1# tifffile.py
2
3# Copyright (c) 2008-2021, Christoph Gohlke
4# All rights reserved.
5#
6# Redistribution and use in source and binary forms, with or without
7# modification, are permitted provided that the following conditions are met:
8#
9# 1. Redistributions of source code must retain the above copyright notice,
10#    this list of conditions and the following disclaimer.
11#
12# 2. Redistributions in binary form must reproduce the above copyright notice,
13#    this list of conditions and the following disclaimer in the documentation
14#    and/or other materials provided with the distribution.
15#
16# 3. Neither the name of the copyright holder nor the names of its
17#    contributors may be used to endorse or promote products derived from
18#    this software without specific prior written permission.
19#
20# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
21# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
22# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
23# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
24# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
25# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
26# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
27# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
28# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
29# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
30# POSSIBILITY OF SUCH DAMAGE.
31
32"""Read and write TIFF files.
33
34Tifffile is a Python library to
35
36(1) store numpy arrays in TIFF (Tagged Image File Format) files, and
37(2) read image and metadata from TIFF-like files used in bioimaging.
38
39Image and metadata can be read from TIFF, BigTIFF, OME-TIFF, STK, LSM, SGI,
40NIHImage, ImageJ, MicroManager, FluoView, ScanImage, SEQ, GEL, SVS, SCN, SIS,
41ZIF (Zoomable Image File Format), QPTIFF (QPI), NDPI, and GeoTIFF files.
42
43Image data can be read as numpy arrays or zarr arrays/groups from strips,
44tiles, pages (IFDs), SubIFDs, higher order series, and pyramidal levels.
45
46Numpy arrays can be written to TIFF, BigTIFF, OME-TIFF, and ImageJ hyperstack
47compatible files in multi-page, volumetric, pyramidal, memory-mappable, tiled,
48predicted, or compressed form.
49
50A subset of the TIFF specification is supported, mainly 8, 16, 32 and 64-bit
51integer, 16, 32 and 64-bit float, grayscale and multi-sample images.
52Specifically, CCITT and OJPEG compression, chroma subsampling without JPEG
53compression, color space transformations, samples with differing types, or
54IPTC and XMP metadata are not implemented.
55
56TIFF, the Tagged Image File Format, was created by the Aldus Corporation and
57Adobe Systems Incorporated. BigTIFF allows for files larger than 4 GB.
58STK, LSM, FluoView, SGI, SEQ, GEL, QPTIFF, NDPI, SCN, SVS, ZIF, and OME-TIFF,
59are custom extensions defined by Molecular Devices (Universal Imaging
60Corporation), Carl Zeiss MicroImaging, Olympus, Silicon Graphics International,
61Media Cybernetics, Molecular Dynamics, PerkinElmer, Hamamatsu, Leica,
62ObjectivePathology, and the Open Microscopy Environment consortium,
63respectively.
64
65For command line usage run ``python -m tifffile --help``
66
67:Author:
68  `Christoph Gohlke <https://www.lfd.uci.edu/~gohlke/>`_
69
70:Organization:
71  Laboratory for Fluorescence Dynamics, University of California, Irvine
72
73:License: BSD 3-Clause
74
75:Version: 2021.8.30
76
77Requirements
78------------
79This release has been tested with the following requirements and dependencies
80(other versions may work):
81
82* `CPython 3.7.9, 3.8.10, 3.9.7 64-bit <https://www.python.org>`_
83* `Numpy 1.20.3 <https://pypi.org/project/numpy/>`_
84* `Imagecodecs 2021.8.26  <https://pypi.org/project/imagecodecs/>`_
85  (required only for encoding or decoding LZW, JPEG, etc.)
86* `Matplotlib 3.4.3 <https://pypi.org/project/matplotlib/>`_
87  (required only for plotting)
88* `Lxml 4.6.3 <https://pypi.org/project/lxml/>`_
89  (required only for validating and printing XML)
90* `Zarr 2.9.4 <https://pypi.org/project/zarr/>`_
91  (required only for opening zarr storage)
92
93Revisions
94---------
952021.8.30
96    Pass 4723 tests.
97    Fix horizontal differencing with non-native byte order.
98    Fix multi-threaded access of memory-mappable, multi-page Zarr stores (#67).
992021.8.8
100    Fix tag offset and valueoffset for NDPI > 4 GB (#96).
1012021.7.30
102    Deprecate first parameter to TiffTag.overwrite (no longer required).
103    TiffTag init API change (breaking).
104    Detect Ventana BIF series and warn that tiles are not stitched.
105    Enable reading PreviewImage from RAW formats (#93, #94).
106    Work around numpy.ndarray.tofile is very slow for non-contiguous arrays.
107    Fix issues with PackBits compression (requires imagecodecs 2021.7.30).
1082021.7.2
109    Decode complex integer images found in SAR GeoTIFF.
110    Support reading NDPI with JPEG-XR compression.
111    Deprecate TiffWriter RGB auto-detection, except for RGB24/48 and RGBA32/64.
1122021.6.14
113    Set stacklevel for deprecation warnings (#89).
114    Fix svs_description_metadata for SVS with double header (#88, breaking).
115    Fix reading JPEG compressed CMYK images.
116    Support ALT_JPEG and JPEG_2000_LOSSY compression found in Bio-Formats.
117    Log warning if TiffWriter auto-detects RGB mode (specify photometric).
1182021.6.6
119    Fix TIFF.COMPESSOR typo (#85).
120    Round resolution numbers that do not fit in 64-bit rationals (#81).
121    Add support for JPEG XL compression.
122    Add numcodecs compatible TIFF codec.
123    Rename ZarrFileStore to ZarrFileSequenceStore (breaking).
124    Add method to export fsspec ReferenceFileSystem from ZarrFileStore.
125    Fix fsspec ReferenceFileSystem v1 for multifile series.
126    Fix creating OME-TIFF with micron character in OME-XML.
1272021.4.8
128    Fix reading OJPEG with wrong photometric or samplesperpixel tags (#75).
129    Fix fsspec ReferenceFileSystem v1 and JPEG compression.
130    Use TiffTagRegistry for NDPI_TAGS, EXIF_TAGS, GPS_TAGS, IOP_TAGS constants.
131    Make TIFF.GEO_KEYS an Enum (breaking).
1322021.3.31
133    Use JPEG restart markers as tile offsets in NDPI.
134    Support version 1 and more codecs in fsspec ReferenceFileSystem (untested).
1352021.3.17
136    Fix regression reading multi-file OME-TIFF with missing files (#72).
137    Fix fsspec ReferenceFileSystem with non-native byte order (#56).
1382021.3.16
139    TIFF is no longer a defended trademark.
140    Add method to export fsspec ReferenceFileSystem from ZarrTiffStore (#56).
1412021.3.5
142    Preliminary support for EER format (#68).
143    Do not warn about unknown compression (#68).
1442021.3.4
145    Fix reading multi-file, multi-series OME-TIFF (#67).
146    Detect ScanImage 2021 files (#46).
147    Shape new version ScanImage series according to metadata (breaking).
148    Remove Description key from TiffFile.scanimage_metadata dict (breaking).
149    Also return ScanImage version from read_scanimage_metadata (breaking).
150    Fix docstrings.
1512021.2.26
152    Squeeze axes of LSM series by default (breaking).
153    Add option to preserve single dimensions when reading from series (WIP).
154    Do not allow appending to OME-TIFF files.
155    Fix reading STK files without name attribute in metadata.
156    Make TIFF constants multi-thread safe and pickleable (#64).
157    Add detection of NDTiffStorage MajorVersion to read_micromanager_metadata.
158    Support ScanImage v4 files in read_scanimage_metadata.
1592021.2.1
160    Fix multi-threaded access of ZarrTiffStores using same TiffFile instance.
161    Use fallback zlib and lzma codecs with imagecodecs lite builds.
162    Open Olympus and Panasonic RAW files for parsing, albeit not supported.
163    Support X2 and X4 differencing found in DNG.
164    Support reading JPEG_LOSSY compression found in DNG.
1652021.1.14
166    Try ImageJ series if OME series fails (#54)
167    Add option to use pages as chunks in ZarrFileStore (experimental).
168    Fix reading from file objects with no readinto function.
1692021.1.11
170    Fix test errors on PyPy.
171    Fix decoding bitorder with imagecodecs >= 2021.1.11.
1722021.1.8
173    Decode float24 using imagecodecs >= 2021.1.8.
174    Consolidate reading of segments if possible.
1752020.12.8
176    Fix corrupted ImageDescription in multi shaped series if buffer too small.
177    Fix libtiff warning that ImageDescription contains null byte in value.
178    Fix reading invalid files using JPEG compression with palette colorspace.
1792020.12.4
180    Fix reading some JPEG compressed CFA images.
181    Make index of SubIFDs a tuple.
182    Pass through FileSequence.imread arguments in imread.
183    Do not apply regex flags to FileSequence axes patterns (breaking).
1842020.11.26
185    Add option to pass axes metadata to ImageJ writer.
186    Pad incomplete tiles passed to TiffWriter.write (#38).
187    Split TiffTag constructor (breaking).
188    Change TiffTag.dtype to TIFF.DATATYPES (breaking).
189    Add TiffTag.overwrite method.
190    Add script to change ImageDescription in files.
191    Add TiffWriter.overwrite_description method (WIP).
1922020.11.18
193    Support writing SEPARATED color space (#37).
194    Use imagecodecs.deflate codec if available.
195    Fix SCN and NDPI series with Z dimensions.
196    Add TiffReader alias for TiffFile.
197    TiffPage.is_volumetric returns True if ImageDepth > 1.
198    Zarr store getitem returns numpy arrays instead of bytes.
1992020.10.1
200    Formally deprecate unused TiffFile parameters (scikit-image #4996).
2012020.9.30
202    Allow to pass additional arguments to compression codecs.
203    Deprecate TiffWriter.save method (use TiffWriter.write).
204    Deprecate TiffWriter.save compress parameter (use compression).
205    Remove multifile parameter from TiffFile (breaking).
206    Pass all is_flag arguments from imread to TiffFile.
207    Do not byte-swap JPEG2000, WEBP, PNG, JPEGXR segments in TiffPage.decode.
2082020.9.29
209    Fix reading files produced by ScanImage > 2015 (#29).
2102020.9.28
211    Derive ZarrStore from MutableMapping.
212    Support zero shape ZarrTiffStore.
213    Fix ZarrFileStore with non-TIFF files.
214    Fix ZarrFileStore with missing files.
215    Cache one chunk in ZarrFileStore.
216    Keep track of already opened files in FileCache.
217    Change parse_filenames function to return zero-based indices.
218    Remove reopen parameter from asarray (breaking).
219    Rename FileSequence.fromfile to imread (breaking).
2202020.9.22
221    Add experimental zarr storage interface (WIP).
222    Remove unused first dimension from TiffPage.shaped (breaking).
223    Move reading of STK planes to series interface (breaking).
224    Always use virtual frames for ScanImage files.
225    Use DimensionOrder to determine axes order in OmeXml.
226    Enable writing striped volumetric images.
227    Keep complete dataoffsets and databytecounts for TiffFrames.
228    Return full size tiles from Tiffpage.segments.
229    Rename TiffPage.is_sgi property to is_volumetric (breaking).
230    Rename TiffPageSeries.is_pyramid to is_pyramidal (breaking).
231    Fix TypeError when passing jpegtables to non-JPEG decode method (#25).
2322020.9.3
233    Do not write contiguous series by default (breaking).
234    Allow to write to SubIFDs (WIP).
235    Fix writing F-contiguous numpy arrays (#24).
2362020.8.25
237    Do not convert EPICS timeStamp to datetime object.
238    Read incompletely written Micro-Manager image file stack header (#23).
239    Remove tag 51123 values from TiffFile.micromanager_metadata (breaking).
2402020.8.13
241    Use tifffile metadata over OME and ImageJ for TiffFile.series (breaking).
242    Fix writing iterable of pages with compression (#20).
243    Expand error checking of TiffWriter data, dtype, shape, and tile arguments.
2442020.7.24
245    Parse nested OmeXml metadata argument (WIP).
246    Do not lazy load TiffFrame JPEGTables.
247    Fix conditionally skipping some tests.
2482020.7.22
249    Do not auto-enable OME-TIFF if description is passed to TiffWriter.save.
250    Raise error writing empty bilevel or tiled images.
251    Allow to write tiled bilevel images.
252    Allow to write multi-page TIFF from iterable of single page images (WIP).
253    Add function to validate OME-XML.
254    Correct Philips slide width and length.
2552020.7.17
256    Initial support for writing OME-TIFF (WIP).
257    Return samples as separate dimension in OME series (breaking).
258    Fix modulo dimensions for multiple OME series.
259    Fix some test errors on big endian systems (#18).
260    Fix BytesWarning.
261    Allow to pass TIFF.PREDICTOR values to TiffWriter.save.
2622020.7.4
263    Deprecate support for Python 3.6 (NEP 29).
264    Move pyramidal subresolution series to TiffPageSeries.levels (breaking).
265    Add parser for SVS, SCN, NDPI, and QPI pyramidal series.
266    Read single-file OME-TIFF pyramids.
267    Read NDPI files > 4 GB (#15).
268    Include SubIFDs in generic series.
269    Preliminary support for writing packed integer arrays (#11, WIP).
270    Read more LSM info subrecords.
271    Fix missing ReferenceBlackWhite tag for YCbCr photometrics.
272    Fix reading lossless JPEG compressed DNG files.
2732020.6.3
274    ...
275
276Refer to the CHANGES file for older revisions.
277
278Notes
279-----
280The API is not stable yet and might change between revisions.
281
282Tested on little-endian platforms only.
283
284Python 32-bit versions are deprecated. Python <= 3.7 are no longer supported.
285
286Tifffile relies on the `imagecodecs <https://pypi.org/project/imagecodecs/>`_
287package for encoding and decoding LZW, JPEG, and other compressed image
288segments.
289
290Several TIFF-like formats do not strictly adhere to the TIFF6 specification,
291some of which allow file or data sizes to exceed the 4 GB limit:
292
293* *BigTIFF* is identified by version number 43 and uses different file
294  header, IFD, and tag structures with 64-bit offsets. It adds more data types.
295  Tifffile can read and write BigTIFF files.
296* *ImageJ* hyperstacks store all image data, which may exceed 4 GB,
297  contiguously after the first IFD. Files > 4 GB contain one IFD only.
298  The size (shape and dtype) of the up to 6-dimensional image data can be
299  determined from the ImageDescription tag of the first IFD, which is Latin-1
300  encoded. Tifffile can read and write ImageJ hyperstacks.
301* *OME-TIFF* stores up to 8-dimensional data in one or multiple TIFF of BigTIFF
302  files. The 8-bit UTF-8 encoded OME-XML metadata found in the ImageDescription
303  tag of the first IFD defines the position of TIFF IFDs in the high
304  dimensional data. Tifffile can read OME-TIFF files, except when the OME-XML
305  metadata are stored in a separate file. Tifffile can write numpy arrays
306  to single-file OME-TIFF.
307* *LSM* stores all IFDs below 4 GB but wraps around 32-bit StripOffsets.
308  The StripOffsets of each series and position require separate unwrapping.
309  The StripByteCounts tag contains the number of bytes for the uncompressed
310  data. Tifffile can read large LSM files.
311* *STK* (MetaMorph Stack) contains additional image planes stored contiguously
312  after the image data of the first page. The total number of planes
313  is equal to the counts of the UIC2tag. Tifffile can read STK files.
314* *NDPI* uses some 64-bit offsets in the file header, IFD, and tag structures.
315  Tag values/offsets can be corrected using high bits stored after IFD
316  structures. Tifffile can read NDPI files > 4 GB.
317  JPEG compressed segments with dimensions >65530 or missing restart markers
318  are not decodable with libjpeg. Tifffile works around this limitation by
319  separately decoding the MCUs between restart markers.
320  BitsPerSample, SamplesPerPixel, and PhotometricInterpretation tags may
321  contain wrong values, which can be corrected using the value of tag 65441.
322* *Philips* TIFF slides store wrong ImageWidth and ImageLength tag values for
323  tiled pages. The values can be corrected using the DICOM_PIXEL_SPACING
324  attributes of the XML formatted description of the first page. Tifffile can
325  read Philips slides.
326* *Ventana BIF* slides store tiles and metadata in a BigTIFF container.
327  Tiles may overlap and require stitching based on the TileJointInfo elements
328  in the XMP tag. Tifffile can read BigTIFF and decode individual tiles,
329  but does not perform stitching.
330* *ScanImage* optionally allows corrupt non-BigTIFF files > 2 GB. The values
331  of StripOffsets and StripByteCounts can be recovered using the constant
332  differences of the offsets of IFD and tag values throughout the file.
333  Tifffile can read such files if the image data are stored contiguously in
334  each page.
335* *GeoTIFF* sparse files allow strip or tile offsets and byte counts to be 0.
336  Such segments are implicitly set to 0 or the NODATA value on reading.
337  Tifffile can read GeoTIFF sparse files.
338
339Other libraries for reading scientific TIFF files from Python:
340
341* `Python-bioformats <https://github.com/CellProfiler/python-bioformats>`_
342* `Imread <https://github.com/luispedro/imread>`_
343* `GDAL <https://github.com/OSGeo/gdal/tree/master/gdal/swig/python>`_
344* `OpenSlide-python <https://github.com/openslide/openslide-python>`_
345* `Slideio <https://gitlab.com/bioslide/slideio>`_
346* `PyLibTiff <https://github.com/pearu/pylibtiff>`_
347* `SimpleITK <https://github.com/SimpleITK/SimpleITK>`_
348* `PyLSM <https://launchpad.net/pylsm>`_
349* `PyMca.TiffIO.py <https://github.com/vasole/pymca>`_ (same as fabio.TiffIO)
350* `BioImageXD.Readers <http://www.bioimagexd.net/>`_
351* `CellCognition <https://cellcognition-project.org/>`_
352* `pymimage <https://github.com/ardoi/pymimage>`_
353* `pytiff <https://github.com/FZJ-INM1-BDA/pytiff>`_
354* `ScanImageTiffReaderPython
355  <https://gitlab.com/vidriotech/scanimagetiffreader-python>`_
356* `bigtiff <https://pypi.org/project/bigtiff>`_
357* `Large Image <https://github.com/girder/large_image>`_
358
359Some libraries are using tifffile to write OME-TIFF files:
360
361* `Zeiss Apeer OME-TIFF library
362  <https://github.com/apeer-micro/apeer-ometiff-library>`_
363* `Allen Institute for Cell Science imageio
364  <https://pypi.org/project/aicsimageio>`_
365* `xtiff <https://github.com/BodenmillerGroup/xtiff>`_
366
367Other tools for inspecting and manipulating TIFF files:
368
369* `tifftools <https://github.com/DigitalSlideArchive/tifftools>`_
370* `Tyf <https://github.com/Moustikitos/tyf>`_
371
372References
373----------
374* TIFF 6.0 Specification and Supplements. Adobe Systems Incorporated.
375  https://www.adobe.io/open/standards/TIFF.html
376* TIFF File Format FAQ. https://www.awaresystems.be/imaging/tiff/faq.html
377* The BigTIFF File Format.
378  https://www.awaresystems.be/imaging/tiff/bigtiff.html
379* MetaMorph Stack (STK) Image File Format.
380  http://mdc.custhelp.com/app/answers/detail/a_id/18862
381* Image File Format Description LSM 5/7 Release 6.0 (ZEN 2010).
382  Carl Zeiss MicroImaging GmbH. BioSciences. May 10, 2011
383* The OME-TIFF format.
384  https://docs.openmicroscopy.org/ome-model/latest/
385* UltraQuant(r) Version 6.0 for Windows Start-Up Guide.
386  http://www.ultralum.com/images%20ultralum/pdf/UQStart%20Up%20Guide.pdf
387* Micro-Manager File Formats.
388  https://micro-manager.org/wiki/Micro-Manager_File_Formats
389* ScanImage BigTiff Specification - ScanImage 2019.
390  http://scanimage.vidriotechnologies.com/display/SI2019/
391  ScanImage+BigTiff+Specification
392* ZIF, the Zoomable Image File format. http://zif.photo/
393* GeoTIFF File Format https://gdal.org/drivers/raster/gtiff.html
394* Cloud optimized GeoTIFF.
395  https://github.com/cogeotiff/cog-spec/blob/master/spec.md
396* Tags for TIFF and Related Specifications. Digital Preservation.
397  https://www.loc.gov/preservation/digital/formats/content/tiff_tags.shtml
398* CIPA DC-008-2016: Exchangeable image file format for digital still cameras:
399  Exif Version 2.31.
400  http://www.cipa.jp/std/documents/e/DC-008-Translation-2016-E.pdf
401* The EER (Electron Event Representation) file format.
402  https://github.com/fei-company/EerReaderLib
403* Digital Negative (DNG) Specification. Version 1.4.0.0, June 2012.
404  https://www.adobe.com/content/dam/acom/en/products/photoshop/pdfs/
405  dng_spec_1.4.0.0.pdf
406
407Examples
408--------
409Write a numpy array to a single-page RGB TIFF file:
410
411>>> data = numpy.random.randint(0, 255, (256, 256, 3), 'uint8')
412>>> imwrite('temp.tif', data, photometric='rgb')
413
414Read the image from the TIFF file as numpy array:
415
416>>> image = imread('temp.tif')
417>>> image.shape
418(256, 256, 3)
419
420Write a 3D numpy array to a multi-page, 16-bit grayscale TIFF file:
421
422>>> data = numpy.random.randint(0, 2**12, (64, 301, 219), 'uint16')
423>>> imwrite('temp.tif', data, photometric='minisblack')
424
425Read the whole image stack from the TIFF file as numpy array:
426
427>>> image_stack = imread('temp.tif')
428>>> image_stack.shape
429(64, 301, 219)
430>>> image_stack.dtype
431dtype('uint16')
432
433Read the image from the first page in the TIFF file as numpy array:
434
435>>> image = imread('temp.tif', key=0)
436>>> image.shape
437(301, 219)
438
439Read images from a selected range of pages:
440
441>>> image = imread('temp.tif', key=range(4, 40, 2))
442>>> image.shape
443(18, 301, 219)
444
445Iterate over all pages in the TIFF file and successively read images:
446
447>>> with TiffFile('temp.tif') as tif:
448...     for page in tif.pages:
449...         image = page.asarray()
450
451Get information about the image stack in the TIFF file without reading
452the image data:
453
454>>> tif = TiffFile('temp.tif')
455>>> len(tif.pages)  # number of pages in the file
45664
457>>> page = tif.pages[0]  # get shape and dtype of the image in the first page
458>>> page.shape
459(301, 219)
460>>> page.dtype
461dtype('uint16')
462>>> page.axes
463'YX'
464>>> series = tif.series[0]  # get shape and dtype of the first image series
465>>> series.shape
466(64, 301, 219)
467>>> series.dtype
468dtype('uint16')
469>>> series.axes
470'QYX'
471>>> tif.close()
472
473Inspect the "XResolution" tag from the first page in the TIFF file:
474
475>>> with TiffFile('temp.tif') as tif:
476...     tag = tif.pages[0].tags['XResolution']
477>>> tag.value
478(1, 1)
479>>> tag.name
480'XResolution'
481>>> tag.code
482282
483>>> tag.count
4841
485>>> tag.dtype
486<DATATYPES.RATIONAL: 5>
487
488Iterate over all tags in the TIFF file:
489
490>>> with TiffFile('temp.tif') as tif:
491...     for page in tif.pages:
492...         for tag in page.tags:
493...             tag_name, tag_value = tag.name, tag.value
494
495Overwrite the value of an existing tag, e.g. XResolution:
496
497>>> with TiffFile('temp.tif', mode='r+b') as tif:
498...     _ = tif.pages[0].tags['XResolution'].overwrite((96000, 1000))
499
500Write a floating-point ndarray and metadata using BigTIFF format, tiling,
501compression, and planar storage:
502
503>>> data = numpy.random.rand(2, 5, 3, 301, 219).astype('float32')
504>>> imwrite('temp.tif', data, bigtiff=True, photometric='minisblack',
505...         compression='zlib', planarconfig='separate', tile=(32, 32),
506...         metadata={'axes': 'TZCYX'})
507
508Write a 10 fps time series of volumes with xyz voxel size 2.6755x2.6755x3.9474
509micron^3 to an ImageJ hyperstack formatted TIFF file:
510
511>>> volume = numpy.random.randn(6, 57, 256, 256).astype('float32')
512>>> imwrite('temp.tif', volume, imagej=True, resolution=(1./2.6755, 1./2.6755),
513...         metadata={'spacing': 3.947368, 'unit': 'um', 'finterval': 1/10,
514...                   'axes': 'TZYX'})
515
516Read the volume and metadata from the ImageJ file:
517
518>>> with TiffFile('temp.tif') as tif:
519...     volume = tif.asarray()
520...     axes = tif.series[0].axes
521...     imagej_metadata = tif.imagej_metadata
522>>> volume.shape
523(6, 57, 256, 256)
524>>> axes
525'TZYX'
526>>> imagej_metadata['slices']
52757
528>>> imagej_metadata['frames']
5296
530
531Create an empty TIFF file and write to the memory-mapped numpy array:
532
533>>> memmap_image = memmap(
534...     'temp.tif', shape=(3, 256, 256), photometric='rgb', dtype='float32'
535... )
536>>> memmap_image[1, 255, 255] = 1.0
537>>> memmap_image.flush()
538>>> del memmap_image
539
540Memory-map image data of the first page in the TIFF file:
541
542>>> memmap_image = memmap('temp.tif', page=0)
543>>> memmap_image[1, 255, 255]
5441.0
545>>> del memmap_image
546
547Write two numpy arrays to a multi-series TIFF file:
548
549>>> series0 = numpy.random.randint(0, 255, (32, 32, 3), 'uint8')
550>>> series1 = numpy.random.randint(0, 1023, (4, 256, 256), 'uint16')
551>>> with TiffWriter('temp.tif') as tif:
552...     tif.write(series0, photometric='rgb')
553...     tif.write(series1, photometric='minisblack')
554
555Read the second image series from the TIFF file:
556
557>>> series1 = imread('temp.tif', series=1)
558>>> series1.shape
559(4, 256, 256)
560
561Successively write the frames of one contiguous series to a TIFF file:
562
563>>> data = numpy.random.randint(0, 255, (30, 301, 219), 'uint8')
564>>> with TiffWriter('temp.tif') as tif:
565...     for frame in data:
566...         tif.write(frame, contiguous=True)
567
568Append an image series to the existing TIFF file:
569
570>>> data = numpy.random.randint(0, 255, (301, 219, 3), 'uint8')
571>>> imwrite('temp.tif', data, photometric='rgb', append=True)
572
573Create a TIFF file from a generator of tiles:
574
575>>> data = numpy.random.randint(0, 2**12, (31, 33, 3), 'uint16')
576>>> def tiles(data, tileshape):
577...     for y in range(0, data.shape[0], tileshape[0]):
578...         for x in range(0, data.shape[1], tileshape[1]):
579...             yield data[y : y + tileshape[0], x : x + tileshape[1]]
580>>> imwrite('temp.tif', tiles(data, (16, 16)), tile=(16, 16),
581...         shape=data.shape, dtype=data.dtype, photometric='rgb')
582
583Write two numpy arrays to a multi-series OME-TIFF file:
584
585>>> series0 = numpy.random.randint(0, 255, (32, 32, 3), 'uint8')
586>>> series1 = numpy.random.randint(0, 1023, (4, 256, 256), 'uint16')
587>>> with TiffWriter('temp.ome.tif') as tif:
588...     tif.write(series0, photometric='rgb')
589...     tif.write(series1, photometric='minisblack',
590...              metadata={'axes': 'ZYX', 'SignificantBits': 10,
591...                        'Plane': {'PositionZ': [0.0, 1.0, 2.0, 3.0]}})
592
593Write a tiled, multi-resolution, pyramidal, OME-TIFF file using
594JPEG compression. Sub-resolution images are written to SubIFDs:
595
596>>> data = numpy.arange(1024*1024*3, dtype='uint8').reshape((1024, 1024, 3))
597>>> with TiffWriter('temp.ome.tif', bigtiff=True) as tif:
598...     options = dict(tile=(256, 256), photometric='rgb', compression='jpeg')
599...     tif.write(data, subifds=2, **options)
600...     # save pyramid levels to the two subifds
601...     # in production use resampling to generate sub-resolutions
602...     tif.write(data[::2, ::2], subfiletype=1, **options)
603...     tif.write(data[::4, ::4], subfiletype=1, **options)
604
605Access the image levels in the pyramidal OME-TIFF file:
606
607>>> baseimage = imread('temp.ome.tif')
608>>> second_level = imread('temp.ome.tif', series=0, level=1)
609>>> with TiffFile('temp.ome.tif') as tif:
610...     baseimage = tif.series[0].asarray()
611...     second_level = tif.series[0].levels[1].asarray()
612
613Iterate over and decode single JPEG compressed tiles in the TIFF file:
614
615>>> with TiffFile('temp.ome.tif') as tif:
616...     fh = tif.filehandle
617...     for page in tif.pages:
618...         for index, (offset, bytecount) in enumerate(
619...             zip(page.dataoffsets, page.databytecounts)
620...         ):
621...             fh.seek(offset)
622...             data = fh.read(bytecount)
623...             tile, indices, shape = page.decode(
624...                 data, index, jpegtables=page.jpegtables
625...             )
626
627Use zarr to read parts of the tiled, pyramidal images in the TIFF file:
628
629>>> import zarr
630>>> store = imread('temp.ome.tif', aszarr=True)
631>>> z = zarr.open(store, mode='r')
632>>> z
633<zarr.hierarchy.Group '/' read-only>
634>>> z[0]  # base layer
635<zarr.core.Array '/0' (1024, 1024, 3) uint8 read-only>
636>>> z[0][256:512, 512:768].shape  # read a tile from the base layer
637(256, 256, 3)
638>>> store.close()
639
640Read images from a sequence of TIFF files as numpy array:
641
642>>> imwrite('temp_C001T001.tif', numpy.random.rand(64, 64))
643>>> imwrite('temp_C001T002.tif', numpy.random.rand(64, 64))
644>>> image_sequence = imread(['temp_C001T001.tif', 'temp_C001T002.tif'])
645>>> image_sequence.shape
646(2, 64, 64)
647
648Read an image stack from a series of TIFF files with a file name pattern
649as numpy or zarr arrays:
650
651>>> image_sequence = TiffSequence('temp_C001*.tif', pattern='axes')
652>>> image_sequence.shape
653(1, 2)
654>>> image_sequence.axes
655'CT'
656>>> data = image_sequence.asarray()
657>>> data.shape
658(1, 2, 64, 64)
659>>> with image_sequence.aszarr() as store:
660...     zarr.open(store, mode='r')
661<zarr.core.Array (1, 2, 64, 64) float64 read-only>
662>>> image_sequence.close()
663
664"""
665
666__version__ = '2021.8.30'
667
668__all__ = (
669    'OmeXml',
670    'OmeXmlError',
671    'TIFF',
672    'TiffFile',
673    'TiffFileError',
674    'TiffFrame',
675    'TiffPage',
676    'TiffPageSeries',
677    'TiffReader',
678    'TiffSequence',
679    'TiffTag',
680    'TiffTags',
681    'TiffTagRegistry',
682    'TiffWriter',
683    'ZarrFileSequenceStore',
684    'ZarrStore',
685    'ZarrTiffStore',
686    'imread',
687    'imshow',
688    'imwrite',
689    'lsm2bin',
690    'memmap',
691    'read_micromanager_metadata',
692    'read_scanimage_metadata',
693    'tiff2fsspec',
694    'tiffcomment',
695    # utility classes and functions used by oiffile, czifile, etc.
696    'FileCache',
697    'FileHandle',
698    'FileSequence',
699    'Timer',
700    'askopenfilename',
701    'astype',
702    'create_output',
703    'enumarg',
704    'enumstr',
705    'format_size',
706    'lazyattr',
707    'matlabstr2py',
708    'natural_sorted',
709    'nullfunc',
710    'parse_kwargs',
711    'pformat',
712    'product',
713    'repeat_nd',
714    'reshape_axes',
715    'reshape_nd',
716    'squeeze_axes',
717    'stripnull',
718    'transpose_axes',
719    'update_kwargs',
720    'xml2dict',
721    # deprecated
722    'imsave',
723    '_app_show',
724)
725
726import binascii
727import collections
728import datetime
729import enum
730import glob
731import io
732import json
733import math
734import os
735import re
736import struct
737import sys
738import threading
739import time
740import warnings
741from collections.abc import Iterable, MutableMapping
742from concurrent.futures import ThreadPoolExecutor
743
744import numpy
745
746try:
747    import imagecodecs
748except Exception:
749    imagecodecs = None
750
751# delay import of mmap, pprint, fractions, xml, lxml, matplotlib, tkinter,
752#   logging, subprocess, multiprocessing, tempfile, zipfile, fnmatch
753
754
755def imread(files=None, aszarr=False, **kwargs):
756    """Return image data from TIFF file(s) as numpy array or zarr storage.
757
758    Refer to the TiffFile and TiffSequence classes and their asarray
759    functions for documentation.
760
761    Parameters
762    ----------
763    files : str, path-like, binary stream, or sequence
764        File name, seekable binary stream, glob pattern, or sequence of
765        file names.
766    aszarr : bool
767        If True, return file sequences, series, or single pages as
768        zarr storage instead of numpy array (experimental).
769    kwargs : dict
770        Parameters 'name', 'offset', 'size', and 'is_' flags are passed to
771        TiffFile or TiffSequence.imread.
772        The 'pattern', 'sort', 'container', and 'axesorder' parameters are
773        passed to TiffSequence().
774        Other parameters are passed to the asarray or aszarr functions.
775        The first image series in the file is returned if no arguments are
776        provided.
777
778    Returns
779    -------
780    numpy.ndarray or zarr storage
781        Image data from the specified pages.
782        Zarr storage instances must be closed after use.
783        See TiffPage.asarray for operations that are applied (or not)
784        to the raw data stored in the file.
785
786    """
787    kwargs_file = parse_kwargs(
788        kwargs,
789        'name',
790        'offset',
791        'size',
792        # private
793        '_multifile',
794        '_useframes',
795        # deprecated, ignored
796        # TODO: remove
797        'fastij',
798        'movie',
799        'multifile',
800        'multifile_close',
801        # is_flags
802        *(key for key in kwargs if key[:3] == 'is_'),
803    )
804    kwargs_seq = parse_kwargs(
805        kwargs, 'pattern', 'sort', 'container', 'imread', 'axesorder'
806    )
807
808    if kwargs.get('pages', None) is not None:
809        # TODO: remove
810        if kwargs.get('key', None) is not None:
811            raise TypeError(
812                "the 'pages' and 'key' parameters cannot be used together"
813            )
814        warnings.warn(
815            "imread: the 'pages' parameter is deprecated since 2017.9.29. "
816            "Use the 'key' parameter",
817            DeprecationWarning,
818            stacklevel=2,
819        )
820        kwargs['key'] = kwargs.pop('pages')
821
822    if kwargs_seq.get('container', None) is None:
823        if isinstance(files, str) and ('*' in files or '?' in files):
824            files = glob.glob(files)
825        if not files:
826            raise ValueError('no files found')
827        if (
828            not hasattr(files, 'seek')
829            and not isinstance(files, (str, os.PathLike))
830            and len(files) == 1
831        ):
832            files = files[0]
833
834        if isinstance(files, (str, os.PathLike)) or hasattr(files, 'seek'):
835            with TiffFile(files, **kwargs_file) as tif:
836                if aszarr:
837                    return tif.aszarr(**kwargs)
838                return tif.asarray(**kwargs)
839
840    with TiffSequence(files, **kwargs_seq) as imseq:
841        if aszarr:
842            return imseq.aszarr(**kwargs, **kwargs_file)
843        return imseq.asarray(**kwargs, **kwargs_file)
844
845
846def imwrite(file, data=None, shape=None, dtype=None, **kwargs):
847    """Write numpy array to TIFF file.
848
849    Refer to the TiffWriter class and its write function for documentation.
850
851    A BigTIFF file is created if the data's size is larger than 4 GB minus
852    32 MB (for metadata), and 'bigtiff' is not specified, and 'imagej' or
853    'truncate' are not enabled.
854
855    Parameters
856    ----------
857    file : str, path-like, or binary stream
858        File name or writable binary stream, such as an open file or BytesIO.
859    data : array-like
860        Input image. The last dimensions are assumed to be image depth,
861        length, width, and samples.
862        If None, an empty array of the specified shape and dtype is
863        saved to file.
864        Unless 'byteorder' is specified in 'kwargs', the TIFF file byte order
865        is determined from the data's dtype or the dtype argument.
866    shape : tuple
867        If 'data' is None, shape of an empty array to save to the file.
868    dtype : numpy.dtype
869        If 'data' is None, datatype of an empty array to save to the file.
870    kwargs : dict
871        Parameters 'append', 'byteorder', 'bigtiff', 'imagej', and 'ome',
872        are passed to TiffWriter().
873        Other parameters are passed to TiffWriter.write().
874
875    Returns
876    -------
877    offset, bytecount : tuple or None
878        If the 'returnoffset' argument is True and the image data are written
879        contiguously, return offset and bytecount of image data in the file.
880
881    """
882    tifargs = parse_kwargs(
883        kwargs, 'append', 'bigtiff', 'byteorder', 'imagej', 'ome'
884    )
885    if data is None:
886        dtype = numpy.dtype(dtype)
887        datasize = product(shape) * dtype.itemsize
888        byteorder = dtype.byteorder
889    else:
890        try:
891            datasize = data.nbytes
892            byteorder = data.dtype.byteorder
893        except Exception:
894            datasize = 0
895            byteorder = None
896    bigsize = kwargs.pop('bigsize', 2 ** 32 - 2 ** 25)
897    if (
898        'bigtiff' not in tifargs
899        and datasize > bigsize
900        and not tifargs.get('imagej', False)
901        and not tifargs.get('truncate', False)
902        and not kwargs.get('compression', False)
903        and not kwargs.get('compress', False)  # TODO: remove deprecated
904    ):
905        tifargs['bigtiff'] = True
906    if 'byteorder' not in tifargs:
907        tifargs['byteorder'] = byteorder
908
909    with TiffWriter(file, **tifargs) as tif:
910        result = tif.write(data, shape, dtype, **kwargs)
911    return result
912
913
914def memmap(
915    filename,
916    shape=None,
917    dtype=None,
918    page=None,
919    series=0,
920    level=0,
921    mode='r+',
922    **kwargs,
923):
924    """Return memory-mapped numpy array stored in TIFF file.
925
926    Memory-mapping requires data stored in native byte order, without tiling,
927    compression, predictors, etc.
928    If 'shape' and 'dtype' are provided, existing files are overwritten or
929    appended to depending on the 'append' parameter.
930    Otherwise the image data of a specified page or series in an existing
931    file are memory-mapped. By default, the image data of the first
932    series are memory-mapped.
933    Call flush() to write any changes in the array to the file.
934    Raise ValueError if the image data in the file are not memory-mappable.
935
936    Parameters
937    ----------
938    filename : str or path-like
939        Name of the TIFF file which stores the array.
940    shape : tuple
941        Shape of the empty array.
942    dtype : numpy.dtype
943        Datatype of the empty array.
944    page : int
945        Index of the page which image data to memory-map.
946    series, level : int
947        Index of the page series and pyramid level which image data to
948        memory-map.
949    mode : {'r+', 'r', 'c'}
950        The file open mode. Default is to open existing file for reading and
951        writing ('r+').
952    kwargs : dict
953        Additional parameters passed to imwrite() or TiffFile().
954
955    Returns
956    -------
957    numpy.memmap
958        Image data in TIFF file.
959
960    """
961    if shape is not None and dtype is not None:
962        # create a new, empty array
963        kwargs.update(
964            data=None,
965            shape=shape,
966            dtype=dtype,
967            align=TIFF.ALLOCATIONGRANULARITY,
968            returnoffset=True,
969        )
970        result = imwrite(filename, **kwargs)
971        if result is None:
972            # TODO: fail before creating file or writing data
973            raise ValueError('image data are not memory-mappable')
974        offset = result[0]
975    else:
976        # use existing file
977        with TiffFile(filename, **kwargs) as tif:
978            if page is not None:
979                page = tif.pages[page]
980                if not page.is_memmappable:
981                    raise ValueError('image data are not memory-mappable')
982                offset, _ = page.is_contiguous
983                shape = page.shape
984                dtype = page.dtype
985            else:
986                series = tif.series[series]
987                if series.offset is None:
988                    raise ValueError('image data are not memory-mappable')
989                shape = series.shape
990                dtype = series.dtype
991                offset = series.offset
992            dtype = tif.byteorder + dtype.char
993    return numpy.memmap(filename, dtype, mode, offset, shape, 'C')
994
995
996class lazyattr:
997    """Attribute whose value is computed on first access.
998
999    Lazyattrs are not thread-safe.
1000
1001    """
1002
1003    # TODO: replace with functools.cached_property? requires Python >= 3.8
1004    __slots__ = ('func', '__dict__')
1005
1006    def __init__(self, func):
1007        """Initialize instance from decorated function."""
1008        self.func = func
1009        self.__doc__ = func.__doc__
1010        self.__module__ = func.__module__
1011        self.__name__ = func.__name__
1012        self.__qualname__ = func.__qualname__
1013        # self.lock = threading.RLock()
1014
1015    def __get__(self, instance, owner):
1016        # with self.lock:
1017        if instance is None:
1018            return self
1019        try:
1020            value = self.func(instance)
1021        except AttributeError as exc:
1022            raise RuntimeError(exc)
1023        if value is NotImplemented:
1024            return getattr(super(owner, instance), self.func.__name__)
1025        setattr(instance, self.func.__name__, value)
1026        return value
1027
1028
1029class TiffFileError(Exception):
1030    """Exception to indicate invalid TIFF structure."""
1031
1032
1033class TiffWriter:
1034    """Write numpy arrays to TIFF file.
1035
1036    TiffWriter's main purpose is saving nD numpy array's as TIFF, not to
1037    create any possible TIFF format. Specifically, ExifIFD and GPSIFD tags.
1038
1039    TiffWriter instances must be closed using the 'close' method, which is
1040    automatically called when using the 'with' context manager.
1041
1042    TiffWriter instances are not thread-safe.
1043
1044    """
1045
1046    def __init__(
1047        self,
1048        file,
1049        bigtiff=False,
1050        byteorder=None,
1051        append=False,
1052        imagej=False,
1053        ome=None,
1054    ):
1055        """Open TIFF file for writing.
1056
1057        An empty TIFF file is created if the file does not exist, else the
1058        file is overwritten with an empty TIFF file unless 'append'
1059        is true. Use 'bigtiff=True' when creating files larger than 4 GB.
1060
1061        Parameters
1062        ----------
1063        file : str, path-like, binary stream, or FileHandle
1064            File name or writable binary stream, such as an open file
1065            or BytesIO.
1066        bigtiff : bool
1067            If True, the BigTIFF format is used.
1068        byteorder : {'<', '>', '=', '|'}
1069            The endianness of the data in the file.
1070            By default, this is the system's native byte order.
1071        append : bool
1072            If True and 'file' is an existing standard TIFF file, image data
1073            and tags are appended to the file.
1074            Appending data may corrupt specifically formatted TIFF files
1075            such as OME-TIFF, LSM, STK, ImageJ, or FluoView.
1076        imagej : bool
1077            If True and not 'ome', write an ImageJ hyperstack compatible file.
1078            This format can handle data types uint8, uint16, or float32 and
1079            data shapes up to 6 dimensions in TZCYXS order.
1080            RGB images (S=3 or S=4) must be uint8.
1081            ImageJ's default byte order is big-endian but this implementation
1082            uses the system's native byte order by default.
1083            ImageJ hyperstacks do not support BigTIFF or compression.
1084            The ImageJ file format is undocumented.
1085            When using compression, use ImageJ's Bio-Formats import function.
1086        ome : bool
1087            If True, write an OME-TIFF compatible file. If None (default),
1088            the value is determined from the file name extension, the value of
1089            the 'description' parameter in the first call of the write
1090            function, and the value of 'imagej'.
1091            Refer to the OME model for restrictions of this format.
1092
1093        """
1094        if append:
1095            # determine if file is an existing TIFF file that can be extended
1096            try:
1097                with FileHandle(file, mode='rb', size=0) as fh:
1098                    pos = fh.tell()
1099                    try:
1100                        with TiffFile(fh) as tif:
1101                            if append != 'force' and not tif.is_appendable:
1102                                raise ValueError(
1103                                    'cannot append to file containing metadata'
1104                                )
1105                            byteorder = tif.byteorder
1106                            bigtiff = tif.is_bigtiff
1107                            self._ifdoffset = tif.pages.next_page_offset
1108                    finally:
1109                        fh.seek(pos)
1110            except (OSError, FileNotFoundError):
1111                append = False
1112
1113        if imagej and bigtiff:
1114            warnings.warn(
1115                'TiffWriter: writing nonconformant BigTIFF ImageJ', UserWarning
1116            )
1117
1118        if byteorder in (None, '=', '|'):
1119            byteorder = '<' if sys.byteorder == 'little' else '>'
1120        elif byteorder not in ('<', '>'):
1121            raise ValueError(f'invalid byteorder {byteorder}')
1122
1123        if byteorder == '<':
1124            self.tiff = TIFF.BIG_LE if bigtiff else TIFF.CLASSIC_LE
1125        else:
1126            self.tiff = TIFF.BIG_BE if bigtiff else TIFF.CLASSIC_BE
1127
1128        self._truncate = False
1129        self._metadata = None
1130        self._colormap = None
1131        self._tags = None
1132        self._datashape = None  # shape of data in consecutive pages
1133        self._datadtype = None  # data type
1134        self._dataoffset = None  # offset to data
1135        self._databytecounts = None  # byte counts per plane
1136        self._dataoffsetstag = None  # strip or tile offset tag code
1137        self._descriptiontag = None  # TiffTag for updating comment
1138        self._subifds = 0  # number of subifds
1139        self._subifdslevel = -1  # index of current subifd level
1140        self._subifdsoffsets = []  # offsets to offsets to subifds
1141        self._nextifdoffsets = []  # offsets to offset to next ifd
1142        self._ifdindex = 0  # index of current ifd
1143
1144        # normalized shape of data in consecutive pages
1145        # (pages, separate_samples, depth, length, width, contig_samples)
1146        self._storedshape = None
1147
1148        if append:
1149            self._fh = FileHandle(file, mode='r+b', size=0)
1150            self._fh.seek(0, os.SEEK_END)
1151        else:
1152            self._fh = FileHandle(file, mode='wb', size=0)
1153            self._fh.write({'<': b'II', '>': b'MM'}[byteorder])
1154            if bigtiff:
1155                self._fh.write(struct.pack(byteorder + 'HHH', 43, 8, 0))
1156            else:
1157                self._fh.write(struct.pack(byteorder + 'H', 42))
1158            # first IFD
1159            self._ifdoffset = self._fh.tell()
1160            self._fh.write(struct.pack(self.tiff.offsetformat, 0))
1161
1162        self._ome = None if ome is None else bool(ome)
1163        self._imagej = False if self._ome else bool(imagej)
1164        if self._imagej:
1165            self._ome = False
1166
1167    @property
1168    def filehandle(self):
1169        """Return file handle."""
1170        return self._fh
1171
1172    def write(
1173        self,
1174        data=None,
1175        shape=None,
1176        dtype=None,
1177        photometric=None,
1178        planarconfig=None,
1179        extrasamples=None,
1180        volumetric=False,
1181        tile=None,
1182        contiguous=False,
1183        truncate=False,
1184        align=None,
1185        rowsperstrip=None,
1186        bitspersample=None,
1187        compression=None,
1188        predictor=None,
1189        subsampling=None,
1190        jpegtables=None,
1191        colormap=None,
1192        description=None,
1193        datetime=None,
1194        resolution=None,
1195        subfiletype=0,
1196        software=None,
1197        subifds=None,
1198        metadata={},
1199        extratags=(),
1200        returnoffset=False,
1201        ijmetadata=None,  # deprecated: use metadata
1202        compress=None,  # deprecated: use compression
1203    ):
1204        """Write numpy ndarray to a series of TIFF pages.
1205
1206        The ND image data are written to a series of TIFF pages/IFDs.
1207        By default, metadata in JSON, ImageJ, or OME-XML format are written
1208        to the ImageDescription tag of the first page to describe the series
1209        such that the image data can later be read back as a ndarray of same
1210        shape.
1211
1212        The data shape's last dimensions are assumed to be image depth,
1213        length (height), width, and samples.
1214        If a colormap is provided, the data's dtype must be uint8 or uint16
1215        and the data values are indices into the last dimension of the
1216        colormap.
1217        If 'shape' and 'dtype' are specified instead of 'data', an empty array
1218        is saved. This option cannot be used with compression, predictors,
1219        packed integers, bilevel images, or multiple tiles.
1220        If 'shape', 'dtype', and 'tile' are specified, 'data' must be an
1221        iterable of all tiles in the image.
1222        If 'shape', 'dtype', and 'data' are specified but not 'tile', 'data'
1223        must be an iterable of all single planes in the image.
1224        Image data are written uncompressed in one strip per plane by default.
1225        Dimensions larger than 2 to 4 (depending on photometric mode, planar
1226        configuration, and volumetric mode) are flattened and saved as separate
1227        pages.
1228        If the data size is zero, a single page with shape (0, 0) is saved.
1229        The SampleFormat tag is derived from the data type or dtype.
1230
1231        A UserWarning is logged if RGB colorspace is auto-detected. Specify
1232        the 'photometric' parameter to avoid the warning.
1233
1234        Parameters
1235        ----------
1236        data : numpy.ndarray, iterable of numpy.ndarray, or None
1237            Input image or iterable of tiles or images.
1238            A copy of the image data is made if 'data' is not a C-contiguous
1239            numpy array with the same byteorder as the TIFF file.
1240            Iterables must yield C-contiguous numpy array of TIFF byteorder.
1241            Iterable tiles must match 'dtype' and the shape specified in
1242            'tile'. Incomplete tiles are zero-padded.
1243            Iterable images must match 'dtype' and 'shape[1:]'.
1244        shape : tuple or None
1245            Shape of the empty or iterable data to save.
1246            Use only if 'data' is None or an iterable of tiles or images.
1247        dtype : numpy.dtype or None
1248            Datatype of the empty or iterable data to save.
1249            Use only if 'data' is None or an iterable of tiles or images.
1250        photometric : {MINISBLACK, MINISWHITE, RGB, PALETTE, SEPARATED, CFA}
1251            The color space of the image data according to TIFF.PHOTOMETRIC.
1252            By default, this setting is inferred from the data shape, dtype,
1253            and the value of colormap. Always specify this parameter to avoid
1254            ambiguities.
1255            For CFA images, the CFARepeatPatternDim, CFAPattern, and other
1256            DNG or TIFF/EP tags must be specified in 'extratags' to produce a
1257            valid file.
1258        planarconfig : {CONTIG, SEPARATE}
1259            Specifies if samples are stored interleaved or in separate planes.
1260            By default, this setting is inferred from the data shape.
1261            If this parameter is set, extra samples are used to store grayscale
1262            images.
1263            CONTIG: last dimension contains samples.
1264            SEPARATE: third (or fourth) last dimension contains samples.
1265        extrasamples : tuple of {UNSPECIFIED, ASSOCALPHA, UNASSALPHA}
1266            Defines the interpretation of extra components in pixels.
1267            UNSPECIFIED: no transparency information (default).
1268            ASSOCALPHA: single, true transparency with pre-multiplied color.
1269            UNASSALPHA: independent transparency masks.
1270        volumetric : bool
1271            If True, the SGI ImageDepth tag is used to save volumetric data
1272            in one page. The volumetric format is not officially specified,
1273            and few software can read it. OME and ImageJ formats are not
1274            compatible with volumetric storage.
1275        tile : tuple of int
1276            The shape ([depth,] length, width) of image tiles to write.
1277            If None (default), image data are written in strips.
1278            The tile length and width must be a multiple of 16.
1279            If a tile depth is provided, the SGI ImageDepth and TileDepth
1280            tags are used to save volumetric data.
1281            Tiles cannot be used to write contiguous series, except if tile
1282            matches the data shape.
1283        contiguous : bool
1284            If False (default), save data to a new series.
1285            If True and the data and parameters are compatible with previous
1286            saved ones (same shape, no compression, etc.), the image data
1287            are stored contiguously after the previous one. In that case,
1288            'photometric', 'planarconfig', and 'rowsperstrip' are ignored.
1289            Metadata such as 'description', 'metadata', 'datetime', and
1290            'extratags' are written to the first page of a contiguous series
1291            only. Cannot be used with the OME or ImageJ formats.
1292        truncate : bool
1293            If True, only write the first page of a contiguous series if
1294            possible (uncompressed, contiguous, not tiled).
1295            Other TIFF readers will only be able to read part of the data.
1296            Cannot be used with the OME or ImageJ formats.
1297        align : int
1298            Byte boundary on which to align the image data in the file.
1299            Default 16. Use mmap.ALLOCATIONGRANULARITY for memory-mapped data.
1300            Following contiguous writes are not aligned.
1301        rowsperstrip : int
1302            The number of rows per strip. By default, strips are ~64 KB if
1303            compression is enabled, else rowsperstrip is set to the image
1304            length.
1305        bitspersample : int
1306            Number of bits per sample. By default, this is the number of
1307            bits of the data dtype. Different values for different samples
1308            are not supported. Unsigned integer data are packed into bytes
1309            as tightly as possible. Valid values are 1-8 for uint8, 9-16 for
1310            uint16 and 17-32 for uint32. Cannot be used with compression,
1311            contiguous series, or empty files.
1312        compression : str, (str, int), (str, int, dict)
1313            If None (default), data are written uncompressed.
1314            If a str, one of TIFF.COMPRESSION, e.g. 'JPEG' or 'ZSTD'.
1315            If a tuple, the first item is one of TIFF.COMPRESSION, the
1316            second item is the compression level, and the third item is a dict
1317            of arguments passed to the compression codec.
1318            Compression cannot be used to write contiguous series.
1319            Compressors may require certain data shapes, types or value ranges.
1320            For example, JPEG requires grayscale or RGB(A), uint8 or 12-bit
1321            uint16. JPEG compression is experimental. JPEG markers and TIFF
1322            tags may not match.
1323            Only a limited set of compression shemes are implemented.
1324        predictor : bool or TIFF.PREDICTOR
1325            If True, apply horizontal differencing or floating-point predictor
1326            before compression. Predictors are disabled for 64-bit integers.
1327        subsampling : {(1, 1), (2, 1), (2, 2), (4, 1)}
1328            The horizontal and vertical subsampling factors used for the
1329            chrominance components of images. The default is (2, 2).
1330            Currently applies to JPEG compression of RGB images only.
1331            Images are stored in YCbCr color space.
1332            Segment widths must be a multiple of 8 times the horizontal factor.
1333            Segment lengths and rowsperstrip must be a multiple of 8 times the
1334            vertical factor.
1335        jpegtables : bytes
1336            JPEG quantization and/or Huffman tables. Use for copying
1337            pre-compressed JPEG segments.
1338        colormap : numpy.ndarray
1339            RGB color values for the corresponding data value.
1340            Must be of shape (3, 2**(data.itemsize*8)) and dtype uint16.
1341        description : str or encoded bytes
1342            The subject of the image. Must be 7-bit ASCII. Cannot be used with
1343            the ImageJ or OME formats. Saved with the first page of a series
1344            only.
1345        datetime : datetime, str, or bool
1346            Date and time of image creation in '%Y:%m:%d %H:%M:%S' format or
1347            datetime object. Else if True, the current date and time is used.
1348            Saved with the first page of a series only.
1349        resolution : (float, float[, str]) or ((int, int), (int, int)[, str])
1350            X and Y resolutions in pixels per resolution unit as float or
1351            rational numbers. A third, optional parameter specifies the
1352            resolution unit, which must be None (default for ImageJ),
1353            'INCH' (default), or 'CENTIMETER'.
1354        subfiletype : int
1355            Bitfield to indicate the kind of data. Set bit 0 if the image
1356            is a reduced-resolution version of another image. Set bit 1 if
1357            the image is part of a multi-page image. Set bit 2 if the image
1358            is transparency mask for another image (photometric must be
1359            MASK, SamplesPerPixel and BitsPerSample must be 1).
1360        software : str or bool
1361            Name of the software used to create the file.
1362            If None (default), 'tifffile.py'. Must be 7-bit ASCII.
1363            Saved with the first page of a series only.
1364        subifds : int
1365            Number of child IFDs. If greater than 0, the following 'subifds'
1366            number of series are written as child IFDs of the current
1367            series. The number of IFDs written for each SubIFD level must match
1368            the number of IFDs written for the current series. All pages
1369            written to a certain SubIFD level of the current series must have
1370            the same hash. SubIFDs cannot be used with truncated or ImageJ
1371            files. SubIFDs in OME-TIFF files must be sub-resolutions of the
1372            main IFDs.
1373        metadata : dict
1374            Additional metadata describing the image data, saved along with
1375            shape information in JSON, OME-XML, or ImageJ formats in
1376            ImageDescription or IJMetadata tags.
1377            If None, do not write an ImageDescription tag with shape in JSON
1378            format.
1379            If ImageJ format, values for keys 'Info', 'Labels', 'Ranges',
1380            'LUTs', 'Plot', 'ROI', and 'Overlays' are saved in IJMetadata and
1381            IJMetadataByteCounts tags. Refer to the imagej_metadata_tag
1382            function for valid values.
1383            Refer to the OmeXml class for supported keys when writing OME-TIFF.
1384            Strings must be 7-bit ASCII.
1385            Saved with the first page of a series only.
1386        extratags : sequence of tuples
1387            Additional tags as [(code, dtype, count, value, writeonce)].
1388
1389            code : int
1390                The TIFF tag Id.
1391            dtype : int or str
1392                Data type of items in 'value'. One of TIFF.DATATYPES.
1393            count : int
1394                Number of data values. Not used for string or bytes values.
1395            value : sequence
1396                'Count' values compatible with 'dtype'.
1397                Bytes must contain count values of dtype packed as binary data.
1398            writeonce : bool
1399                If True, the tag is written to the first page of a series only.
1400
1401        returnoffset : bool
1402            If True and the image data in the file are memory-mappable, return
1403            the offset and number of bytes of the image data in the file.
1404
1405        Returns
1406        -------
1407        offset, bytecount : tuple or None
1408            If 'returnoffset' is true and the image data in the file are
1409            memory-mappable, return the offset and number of bytes of the
1410            image data in the file.
1411
1412        """
1413        # TODO: refactor this function
1414        fh = self._fh
1415        byteorder = self.tiff.byteorder
1416
1417        if data is None:
1418            # empty
1419            dataiter = None
1420            datashape = tuple(shape)
1421            datadtype = numpy.dtype(dtype).newbyteorder(byteorder)
1422            datadtypechar = datadtype.char
1423        elif (
1424            shape is not None
1425            and dtype is not None
1426            and hasattr(data, '__iter__')
1427        ):
1428            # iterable pages or tiles
1429            if hasattr(data, '__next__'):
1430                dataiter = data
1431            else:
1432                dataiter = iter(data)
1433            datashape = tuple(shape)
1434            datadtype = numpy.dtype(dtype).newbyteorder(byteorder)
1435            datadtypechar = datadtype.char
1436        elif hasattr(data, '__next__'):
1437            # generator
1438            raise TypeError(
1439                "generators require 'shape' and 'dtype' parameters"
1440            )
1441        else:
1442            # whole image data
1443            # must be C-contiguous numpy array of TIFF byteorder
1444            if hasattr(data, 'dtype'):
1445                data = numpy.asarray(data, byteorder + data.dtype.char, 'C')
1446            else:
1447                datadtype = numpy.dtype(dtype).newbyteorder(byteorder)
1448                data = numpy.asarray(data, datadtype, 'C')
1449
1450            if dtype is not None and dtype != data.dtype:
1451                warnings.warn(
1452                    "TiffWriter: ignoring 'dtype' argument", UserWarning
1453                )
1454            if shape is not None and shape != data.shape:
1455                warnings.warn(
1456                    "TiffWriter: ignoring 'shape' argument", UserWarning
1457                )
1458            dataiter = None
1459            datashape = data.shape
1460            datadtype = data.dtype
1461            datadtypechar = data.dtype.char
1462
1463        returnoffset = returnoffset and datadtype.isnative
1464
1465        if compression is None and compress is not None:
1466            # TODO: remove
1467            warnings.warn(
1468                "TiffWriter: the 'compress' parameter is deprecated "
1469                "since 2020.9.30. Use the 'compression' parameter",
1470                DeprecationWarning,
1471                stacklevel=2,
1472            )
1473            if isinstance(compress, (int, numpy.integer)) and compress > 0:
1474                # ADOBE_DEFLATE
1475                compression = 8, int(compress)
1476                if not 0 < compress <= 9:
1477                    raise ValueError(f'invalid compression level {compress}')
1478            else:
1479                compression = compress
1480            del compress
1481
1482        bilevel = datadtypechar == '?'
1483        if bilevel:
1484            index = -1 if datashape[-1] > 1 else -2
1485            datasize = product(datashape[:index])
1486            if datashape[index] % 8:
1487                datasize *= datashape[index] // 8 + 1
1488            else:
1489                datasize *= datashape[index] // 8
1490        else:
1491            datasize = product(datashape) * datadtype.itemsize
1492
1493        if datasize == 0:
1494            data = None
1495            compression = False
1496            bitspersample = None
1497            if metadata is not None:
1498                truncate = True
1499        elif compression in (None, 0, 1, 'NONE', 'none'):
1500            compression = False
1501
1502        inputshape = datashape
1503
1504        packints = (
1505            bitspersample is not None
1506            and bitspersample != datadtype.itemsize * 8
1507        )
1508
1509        # just append contiguous data if possible
1510        if self._datashape is not None:
1511            if (
1512                not contiguous
1513                or self._datashape[1:] != datashape
1514                or self._datadtype != datadtype
1515                or not numpy.array_equal(colormap, self._colormap)
1516            ):
1517                # incompatible shape, dtype, or colormap
1518                self._write_remaining_pages()
1519
1520                if self._imagej:
1521                    raise ValueError(
1522                        'ImageJ does not support non-contiguous series'
1523                    )
1524                elif self._ome:
1525                    if self._subifdslevel < 0:
1526                        # add image to OME-XML
1527                        self._ome.addimage(
1528                            self._datadtype,
1529                            self._datashape[
1530                                0 if self._datashape[0] != 1 else 1 :
1531                            ],
1532                            self._storedshape,
1533                            **self._metadata,
1534                        )
1535                elif metadata is not None:
1536                    self._write_image_description()
1537                    # description might have been appended to file
1538                    fh.seek(0, os.SEEK_END)
1539
1540                if self._subifds:
1541                    if self._truncate or truncate:
1542                        raise ValueError(
1543                            'SubIFDs cannot be used with truncated series'
1544                        )
1545                    self._subifdslevel += 1
1546                    if self._subifdslevel == self._subifds:
1547                        # done with writing SubIFDs
1548                        self._nextifdoffsets = []
1549                        self._subifdsoffsets = []
1550                        self._subifdslevel = -1
1551                        self._subifds = 0
1552                        self._ifdindex = 0
1553                    elif subifds:
1554                        raise ValueError(
1555                            'SubIFDs in SubIFDs are not supported'
1556                        )
1557
1558                self._datashape = None
1559                self._colormap = None
1560
1561            elif compression or packints or tile:
1562                raise ValueError(
1563                    'contiguous cannot be used with compression, tiles, etc.'
1564                )
1565
1566            else:
1567                # consecutive mode
1568                # write contiguous data, write IFDs/tags later
1569                self._datashape = (self._datashape[0] + 1,) + datashape
1570                offset = fh.tell()
1571                if data is None:
1572                    fh.write_empty(datasize)
1573                else:
1574                    fh.write_array(data)
1575                if returnoffset:
1576                    return offset, datasize
1577                return None
1578
1579        if self._ome is None:
1580            if description is None:
1581                self._ome = '.ome.tif' in fh.name
1582            else:
1583                self._ome = False
1584        self._truncate = False if self._ome else bool(truncate)
1585
1586        if datasize == 0:
1587            # write single placeholder TiffPage for arrays with size=0
1588            datashape = (0, 0)
1589            warnings.warn(
1590                'TiffWriter: writing zero size array to nonconformant TIFF',
1591                UserWarning,
1592            )
1593            # TODO: reconsider this
1594            # raise ValueError('cannot save zero size array')
1595
1596        valueformat = f'{self.tiff.offsetsize}s'
1597        tagnoformat = self.tiff.tagnoformat
1598        offsetformat = self.tiff.offsetformat
1599        offsetsize = self.tiff.offsetsize
1600        tagsize = self.tiff.tagsize
1601
1602        MINISBLACK = TIFF.PHOTOMETRIC.MINISBLACK
1603        MINISWHITE = TIFF.PHOTOMETRIC.MINISWHITE
1604        RGB = TIFF.PHOTOMETRIC.RGB
1605        YCBCR = TIFF.PHOTOMETRIC.YCBCR
1606        PALETTE = TIFF.PHOTOMETRIC.PALETTE
1607        CONTIG = TIFF.PLANARCONFIG.CONTIG
1608        SEPARATE = TIFF.PLANARCONFIG.SEPARATE
1609
1610        # parse input
1611        if photometric is not None:
1612            photometric = enumarg(TIFF.PHOTOMETRIC, photometric)
1613        if planarconfig:
1614            planarconfig = enumarg(TIFF.PLANARCONFIG, planarconfig)
1615        if predictor:
1616            if not isinstance(predictor, bool):
1617                predictor = bool(enumarg(TIFF.PREDICTOR, predictor))
1618        if extrasamples is None:
1619            extrasamples_ = None
1620        else:
1621            extrasamples_ = tuple(
1622                enumarg(TIFF.EXTRASAMPLE, es) for es in sequence(extrasamples)
1623            )
1624
1625        if compression:
1626            if isinstance(compression, (tuple, list)):
1627                if len(compression) == 2:
1628                    compressionargs = {'level': compression[1]}
1629                else:
1630                    compressionargs = dict(compression[2])
1631                    if compression[1] is not None:
1632                        compressionargs['level'] = compression[1]
1633                compression = compression[0]
1634            else:
1635                compressionargs = {}
1636            if isinstance(compression, str):
1637                compression = compression.upper()
1638                if compression == 'ZLIB':
1639                    compression = 8  # ADOBE_DEFLATE
1640            compressiontag = enumarg(TIFF.COMPRESSION, compression)
1641            compression = compressiontag > 1
1642        else:
1643            compression = False
1644            compressiontag = 1
1645
1646        if not compression:
1647            compressionargs = {}
1648            predictor = False  # TODO: support predictors without compression?
1649            predictortag = 1
1650
1651        if predictor:
1652            if compressiontag in (
1653                7,
1654                33003,
1655                33004,
1656                33005,
1657                33007,
1658                34712,
1659                34892,
1660                34933,
1661                34934,
1662                50001,
1663                50002,
1664            ):
1665                # disable predictor for JPEG, JPEG2000, WEBP, PNG, JPEGXR
1666                predictor = False
1667            elif datadtype.kind in 'iu':
1668                if datadtype.itemsize > 4:
1669                    predictor = False  # disable predictor for 64 bit
1670                else:
1671                    predictortag = 2
1672                    predictor = TIFF.PREDICTORS[2]
1673            elif datadtype.kind == 'f':
1674                predictortag = 3
1675                predictor = TIFF.PREDICTORS[3]
1676            else:
1677                raise ValueError(f'cannot apply predictor to {datadtype}')
1678
1679        if self._ome:
1680            if description is not None:
1681                warnings.warn(
1682                    'TiffWriter: not writing description to OME-TIFF',
1683                    UserWarning,
1684                )
1685                description = None
1686            if not isinstance(self._ome, OmeXml):
1687                self._ome = OmeXml(**metadata)
1688            if volumetric or (tile and len(tile) > 2):
1689                raise ValueError('OME-TIFF does not support ImageDepth')
1690            volumetric = False
1691
1692        elif self._imagej:
1693            # if tile is not None or predictor or compression:
1694            #     warnings.warn(
1695            #         'ImageJ does not support tiles, predictors, compression'
1696            #     )
1697            if description is not None:
1698                warnings.warn(
1699                    'TiffWriter: not writing description to ImageJ file',
1700                    UserWarning,
1701                )
1702                description = None
1703            if datadtypechar not in 'BHhf':
1704                raise ValueError(
1705                    f'ImageJ does not support data type {datadtypechar!r}'
1706                )
1707            if volumetric or (tile and len(tile) > 2):
1708                raise ValueError('ImageJ does not support ImageDepth')
1709            volumetric = False
1710            ijrgb = photometric == RGB if photometric else None
1711            if datadtypechar not in 'B':
1712                ijrgb = False
1713            ijshape = imagej_shape(
1714                datashape, ijrgb, metadata.get('axes', None)
1715            )
1716            if ijshape[-1] in (3, 4):
1717                photometric = RGB
1718                if datadtypechar not in 'B':
1719                    raise ValueError(
1720                        'ImageJ does not support '
1721                        f'data type {datadtypechar!r} for RGB'
1722                    )
1723            elif photometric is None:
1724                photometric = MINISBLACK
1725                planarconfig = None
1726            if planarconfig == SEPARATE:
1727                raise ValueError('ImageJ does not support planar images')
1728            planarconfig = CONTIG if ijrgb else None
1729
1730        # verify colormap and indices
1731        if colormap is not None:
1732            if datadtypechar not in 'BH':
1733                raise ValueError('invalid data dtype for palette mode')
1734            colormap = numpy.asarray(colormap, dtype=byteorder + 'H')
1735            if colormap.shape != (3, 2 ** (datadtype.itemsize * 8)):
1736                raise ValueError('invalid color map shape')
1737            self._colormap = colormap
1738
1739        if tile:
1740            # verify tile shape
1741            tile = tuple(int(i) for i in tile[:3])
1742            if (
1743                len(tile) < 2
1744                or tile[-1] % 16
1745                or tile[-2] % 16
1746                or any(i < 1 for i in tile)
1747            ):
1748                raise ValueError('invalid tile shape')
1749            if volumetric and len(tile) == 2:
1750                tile = (1,) + tile
1751            volumetric = len(tile) == 3
1752        else:
1753            tile = ()
1754            volumetric = bool(volumetric)
1755
1756        # normalize data shape to 5D or 6D, depending on volume:
1757        #   (pages, separate_samples, [depth,] length, width, contig_samples)
1758        storedshape = reshape_nd(
1759            datashape, TIFF.PHOTOMETRIC_SAMPLES.get(photometric, 2)
1760        )
1761        shape = storedshape
1762        ndim = len(storedshape)
1763
1764        samplesperpixel = 1
1765        extrasamples = 0
1766
1767        if volumetric and ndim < 3:
1768            volumetric = False
1769
1770        if colormap is not None:
1771            photometric = PALETTE
1772            planarconfig = None
1773
1774        if photometric is None:
1775            deprecate = False
1776            photometric = MINISBLACK
1777            if bilevel:
1778                photometric = MINISWHITE
1779            elif planarconfig == CONTIG:
1780                if ndim > 2 and shape[-1] in (3, 4):
1781                    photometric = RGB
1782                    deprecate = datadtypechar not in 'BH'
1783            elif planarconfig == SEPARATE:
1784                if volumetric and ndim > 3 and shape[-4] in (3, 4):
1785                    photometric = RGB
1786                    deprecate = True
1787                elif ndim > 2 and shape[-3] in (3, 4):
1788                    photometric = RGB
1789                    deprecate = True
1790            elif ndim > 2 and shape[-1] in (3, 4):
1791                photometric = RGB
1792                planarconfig = CONTIG
1793                deprecate = datadtypechar not in 'BH'
1794            elif self._imagej or self._ome:
1795                photometric = MINISBLACK
1796                planarconfig = None
1797            elif volumetric and ndim > 3 and shape[-4] in (3, 4):
1798                photometric = RGB
1799                planarconfig = SEPARATE
1800                deprecate = True
1801            elif ndim > 2 and shape[-3] in (3, 4):
1802                photometric = RGB
1803                planarconfig = SEPARATE
1804                deprecate = True
1805
1806            if deprecate:
1807                if planarconfig == CONTIG:
1808                    msg = 'contiguous samples', "parameter is"
1809                else:
1810                    msg = (
1811                        'separate component planes',
1812                        "and 'planarconfig' parameters are",
1813                    )
1814                warnings.warn(
1815                    f"TiffWriter: data with shape {datashape} and dtype "
1816                    f"'{datadtype}'' are stored as RGB with {msg[0]}. "
1817                    "Future versions will store such data as MINISBLACK in "
1818                    "separate pages by default unless the 'photometric' "
1819                    f"{msg[1]} specified.",
1820                    DeprecationWarning,
1821                    stacklevel=2,
1822                )
1823                del msg
1824            del deprecate
1825
1826        del datashape
1827        photometricsamples = TIFF.PHOTOMETRIC_SAMPLES[photometric]
1828
1829        if planarconfig and len(shape) <= (3 if volumetric else 2):
1830            # TODO: raise error?
1831            planarconfig = None
1832            if photometricsamples > 1:
1833                photometric = MINISBLACK
1834
1835        if photometricsamples > 1:
1836            if len(shape) < 3:
1837                raise ValueError(f'not a {photometric!r} image')
1838            if len(shape) < 4:
1839                volumetric = False
1840            if planarconfig is None:
1841                if photometric == RGB:
1842                    samples_ = (photometricsamples, 4)  # allow common alpha
1843                else:
1844                    samples_ = (photometricsamples,)
1845                if shape[-1] in samples_:
1846                    planarconfig = CONTIG
1847                elif shape[-4 if volumetric else -3] in samples_:
1848                    planarconfig = SEPARATE
1849                elif shape[-1] > shape[-4 if volumetric else -3]:
1850                    # TODO: deprecated this?
1851                    planarconfig = SEPARATE
1852                else:
1853                    planarconfig = CONTIG
1854            if planarconfig == CONTIG:
1855                storedshape = (-1, 1) + shape[(-4 if volumetric else -3) :]
1856                samplesperpixel = storedshape[-1]
1857            else:
1858                storedshape = (
1859                    (-1,) + shape[(-4 if volumetric else -3) :] + (1,)
1860                )
1861                samplesperpixel = storedshape[1]
1862            if samplesperpixel > photometricsamples:
1863                extrasamples = samplesperpixel - photometricsamples
1864
1865        elif photometric == TIFF.PHOTOMETRIC.CFA:
1866            if len(shape) != 2:
1867                raise ValueError('invalid CFA image')
1868            volumetric = False
1869            planarconfig = None
1870            storedshape = (-1, 1) + shape[-2:] + (1,)
1871            # if all(et[0] != 50706 for et in extratags):
1872            #     raise ValueError('must specify DNG tags for CFA image')
1873
1874        elif planarconfig and len(shape) > (3 if volumetric else 2):
1875            if planarconfig == CONTIG:
1876                storedshape = (-1, 1) + shape[(-4 if volumetric else -3) :]
1877                samplesperpixel = storedshape[-1]
1878            else:
1879                storedshape = (
1880                    (-1,) + shape[(-4 if volumetric else -3) :] + (1,)
1881                )
1882                samplesperpixel = storedshape[1]
1883            extrasamples = samplesperpixel - 1
1884
1885        else:
1886            planarconfig = None
1887            while len(shape) > 2 and shape[-1] == 1:
1888                shape = shape[:-1]  # remove trailing 1s
1889            if len(shape) < 3:
1890                volumetric = False
1891            if extrasamples_ is None:
1892                storedshape = (
1893                    (-1, 1) + shape[(-3 if volumetric else -2) :] + (1,)
1894                )
1895            else:
1896                storedshape = (-1, 1) + shape[(-4 if volumetric else -3) :]
1897                samplesperpixel = storedshape[-1]
1898                extrasamples = samplesperpixel - 1
1899
1900        if subfiletype & 0b100:
1901            # FILETYPE_MASK
1902            if not (
1903                bilevel and samplesperpixel == 1 and photometric in (0, 1, 4)
1904            ):
1905                raise ValueError('invalid SubfileType MASK')
1906            photometric = TIFF.PHOTOMETRIC.MASK
1907
1908        packints = False
1909        if bilevel:
1910            if bitspersample is not None and bitspersample != 1:
1911                raise ValueError('bitspersample must be 1 for bilevel')
1912            bitspersample = 1
1913        elif compressiontag == 7 and datadtype == 'uint16':
1914            if bitspersample is not None and bitspersample != 12:
1915                raise ValueError(
1916                    'bitspersample must be 12 for JPEG compressed uint16'
1917                )
1918            bitspersample = 12  # use 12-bit JPEG compression
1919        elif bitspersample is None:
1920            bitspersample = datadtype.itemsize * 8
1921        elif (
1922            datadtype.kind != 'u' or datadtype.itemsize > 4
1923        ) and bitspersample != datadtype.itemsize * 8:
1924            raise ValueError('bitspersample does not match dtype')
1925        elif not (
1926            bitspersample > {1: 0, 2: 8, 4: 16}[datadtype.itemsize]
1927            and bitspersample <= datadtype.itemsize * 8
1928        ):
1929            raise ValueError('bitspersample out of range of dtype')
1930        elif compression:
1931            if bitspersample != datadtype.itemsize * 8:
1932                raise ValueError(
1933                    'bitspersample cannot be used with compression'
1934                )
1935        elif bitspersample != datadtype.itemsize * 8:
1936            packints = True
1937
1938        # normalize storedshape to 6D
1939        if len(storedshape) not in (5, 6):
1940            raise RuntimeError('len(storedshape) not in (5, 6)')
1941        if len(storedshape) == 5:
1942            storedshape = storedshape[:2] + (1,) + storedshape[2:]
1943        if storedshape[0] == -1:
1944            s0 = product(storedshape[1:])
1945            s0 = 1 if s0 == 0 else product(inputshape) // s0
1946            storedshape = (s0,) + storedshape[1:]
1947        try:
1948            data = data.reshape(storedshape)
1949        except AttributeError:
1950            pass  # data is None or iterator
1951
1952        if photometric == PALETTE:
1953            if (
1954                samplesperpixel != 1
1955                or extrasamples
1956                or storedshape[1] != 1
1957                or storedshape[-1] != 1
1958            ):
1959                raise ValueError('invalid data shape for palette mode')
1960
1961        if photometric == RGB and samplesperpixel == 2:
1962            raise ValueError('not a RGB image (samplesperpixel=2)')
1963
1964        tags = []  # list of (code, ifdentry, ifdvalue, writeonce)
1965
1966        if tile:
1967            tagbytecounts = 325  # TileByteCounts
1968            tagoffsets = 324  # TileOffsets
1969        else:
1970            tagbytecounts = 279  # StripByteCounts
1971            tagoffsets = 273  # StripOffsets
1972        self._dataoffsetstag = tagoffsets
1973
1974        def pack(fmt, *val):
1975            if fmt[0] not in '<>':
1976                fmt = byteorder + fmt
1977            return struct.pack(fmt, *val)
1978
1979        def addtag(code, dtype, count, value, writeonce=False):
1980            # compute ifdentry & ifdvalue bytes from code, dtype, count, value
1981            # append (code, ifdentry, ifdvalue, writeonce) to tags list
1982            if not isinstance(code, int):
1983                code = TIFF.TAGS[code]
1984            try:
1985                datatype = dtype
1986                dataformat = TIFF.DATA_FORMATS[datatype][-1]
1987            except KeyError as exc:
1988                try:
1989                    dataformat = dtype
1990                    if dataformat[0] in '<>':
1991                        dataformat = dataformat[1:]
1992                    datatype = TIFF.DATA_DTYPES[dataformat]
1993                except (KeyError, TypeError):
1994                    raise ValueError(f'unknown dtype {dtype}') from exc
1995            del dtype
1996
1997            rawcount = count
1998            if datatype == 2:
1999                # string
2000                if isinstance(value, str):
2001                    # enforce 7-bit ASCII on Unicode strings
2002                    try:
2003                        value = value.encode('ascii')
2004                    except UnicodeEncodeError as exc:
2005                        raise ValueError(
2006                            'TIFF strings must be 7-bit ASCII'
2007                        ) from exc
2008                elif not isinstance(value, bytes):
2009                    raise ValueError('TIFF strings must be 7-bit ASCII')
2010                if len(value) == 0 or value[-1] != b'\x00':
2011                    value += b'\x00'
2012                count = len(value)
2013                if code == 270:
2014                    self._descriptiontag = TiffTag(
2015                        self, 0, 270, 2, count, None, 0
2016                    )
2017                    rawcount = value.find(b'\x00\x00')
2018                    if rawcount < 0:
2019                        rawcount = count
2020                    else:
2021                        # length of string without buffer
2022                        rawcount = max(offsetsize + 1, rawcount + 1)
2023                        rawcount = min(count, rawcount)
2024                else:
2025                    rawcount = count
2026                value = (value,)
2027
2028            elif isinstance(value, bytes):
2029                # packed binary data
2030                itemsize = struct.calcsize(dataformat)
2031                if len(value) % itemsize:
2032                    raise ValueError('invalid packed binary data')
2033                count = len(value) // itemsize
2034                rawcount = count
2035
2036            if datatype in (5, 10):  # rational
2037                count *= 2
2038                dataformat = dataformat[-1]
2039
2040            ifdentry = [
2041                pack('HH', code, datatype),
2042                pack(offsetformat, rawcount),
2043            ]
2044
2045            ifdvalue = None
2046            if struct.calcsize(dataformat) * count <= offsetsize:
2047                # value(s) can be written directly
2048                if isinstance(value, bytes):
2049                    ifdentry.append(pack(valueformat, value))
2050                elif count == 1:
2051                    if isinstance(value, (tuple, list, numpy.ndarray)):
2052                        value = value[0]
2053                    ifdentry.append(pack(valueformat, pack(dataformat, value)))
2054                else:
2055                    ifdentry.append(
2056                        pack(valueformat, pack(f'{count}{dataformat}', *value))
2057                    )
2058            else:
2059                # use offset to value(s)
2060                ifdentry.append(pack(offsetformat, 0))
2061                if isinstance(value, bytes):
2062                    ifdvalue = value
2063                elif isinstance(value, numpy.ndarray):
2064                    if value.size != count:
2065                        raise RuntimeError('value.size != count')
2066                    if value.dtype.char != dataformat:
2067                        raise RuntimeError('value.dtype.char != dtype')
2068                    ifdvalue = value.tobytes()
2069                elif isinstance(value, (tuple, list)):
2070                    ifdvalue = pack(f'{count}{dataformat}', *value)
2071                else:
2072                    ifdvalue = pack(dataformat, value)
2073            tags.append((code, b''.join(ifdentry), ifdvalue, writeonce))
2074
2075        def rational(arg):
2076            # return numerator and denominator from float or two integers
2077            from fractions import Fraction  # delayed import
2078
2079            try:
2080                f = Fraction.from_float(arg)
2081            except TypeError:
2082                f = Fraction(arg[0], arg[1])
2083            try:
2084                numerator, denominator = f.as_integer_ratio()
2085            except AttributeError:
2086                # Python 3.7
2087                f = f.limit_denominator(4294967294)
2088                numerator, denominator = f.numerator, f.denominator
2089            if numerator > 4294967295 or denominator > 4294967295:
2090                s = 4294967295 / max(numerator, denominator)
2091                numerator = round(numerator * s)
2092                denominator = round(denominator * s)
2093            return numerator, denominator
2094
2095        if description is not None:
2096            # ImageDescription: user provided description
2097            addtag(270, 2, 0, description, writeonce=True)
2098
2099        # write shape and metadata to ImageDescription
2100        self._metadata = {} if not metadata else metadata.copy()
2101        if self._ome:
2102            if len(self._ome.images) == 0:
2103                description = ''  # rewritten later at end of file
2104            else:
2105                description = None
2106        elif self._imagej:
2107            if ijmetadata is None:
2108                ijmetadata = parse_kwargs(
2109                    self._metadata,
2110                    'Info',
2111                    'Labels',
2112                    'Ranges',
2113                    'LUTs',
2114                    'Plot',
2115                    'ROI',
2116                    'Overlays',
2117                    'info',
2118                    'labels',
2119                    'ranges',
2120                    'luts',
2121                    'plot',
2122                    'roi',
2123                    'overlays',
2124                )
2125            else:
2126                # TODO: remove
2127                warnings.warn(
2128                    "TiffWriter: the 'ijmetadata' parameter is deprecated "
2129                    "since 2020.5.5. Use the 'metadata' parameter",
2130                    DeprecationWarning,
2131                    stacklevel=2,
2132                )
2133            for t in imagej_metadata_tag(ijmetadata, byteorder):
2134                addtag(*t)
2135            description = imagej_description(
2136                inputshape,
2137                storedshape[-1] in (3, 4),
2138                self._colormap is not None,
2139                **self._metadata,
2140            )
2141            description += '\x00' * 64  # add buffer for in-place update
2142        elif metadata or metadata == {}:
2143            if self._truncate:
2144                self._metadata.update(truncated=True)
2145            description = json_description(inputshape, **self._metadata)
2146            description += '\x00' * 16  # add buffer for in-place update
2147        # elif metadata is None and self._truncate:
2148        #     raise ValueError('cannot truncate without writing metadata')
2149        else:
2150            description = None
2151
2152        if description is not None:
2153            description = description.encode('ascii')
2154            addtag(270, 2, 0, description, writeonce=True)
2155        del description
2156
2157        if software is None:
2158            software = 'tifffile.py'
2159        if software:
2160            addtag(305, 2, 0, software, writeonce=True)
2161        if datetime:
2162            if isinstance(datetime, str):
2163                if len(datetime) != 19 or datetime[16] != ':':
2164                    raise ValueError('invalid datetime string')
2165            else:
2166                try:
2167                    datetime = datetime.strftime('%Y:%m:%d %H:%M:%S')
2168                except AttributeError:
2169                    datetime = self._now().strftime('%Y:%m:%d %H:%M:%S')
2170            addtag(306, 2, 0, datetime, writeonce=True)
2171        addtag(259, 3, 1, compressiontag)  # Compression
2172        if compressiontag == 34887:
2173            # LERC without additional compression
2174            addtag(50674, 4, 2, (4, 0))
2175        if predictor:
2176            addtag(317, 3, 1, predictortag)
2177        addtag(256, 4, 1, storedshape[-2])  # ImageWidth
2178        addtag(257, 4, 1, storedshape[-3])  # ImageLength
2179        if tile:
2180            addtag(322, 4, 1, tile[-1])  # TileWidth
2181            addtag(323, 4, 1, tile[-2])  # TileLength
2182        if volumetric:
2183            addtag(32997, 4, 1, storedshape[-4])  # ImageDepth
2184            if tile:
2185                addtag(32998, 4, 1, tile[0])  # TileDepth
2186        if subfiletype:
2187            addtag(254, 4, 1, subfiletype)  # NewSubfileType
2188        if (subifds or self._subifds) and self._subifdslevel < 0:
2189            if self._subifds:
2190                subifds = self._subifds
2191            else:
2192                try:
2193                    self._subifds = subifds = int(subifds)
2194                except TypeError:
2195                    # allow TiffPage.subifds tuple
2196                    self._subifds = subifds = len(subifds)
2197            addtag(330, 18 if offsetsize > 4 else 13, subifds, [0] * subifds)
2198        if not bilevel and not datadtype.kind == 'u':
2199            sampleformat = {'u': 1, 'i': 2, 'f': 3, 'c': 6}[datadtype.kind]
2200            addtag(339, 3, samplesperpixel, (sampleformat,) * samplesperpixel)
2201        if colormap is not None:
2202            addtag(320, 3, colormap.size, colormap)
2203        addtag(277, 3, 1, samplesperpixel)
2204        if bilevel:
2205            pass
2206        elif planarconfig and samplesperpixel > 1:
2207            addtag(284, 3, 1, planarconfig.value)  # PlanarConfiguration
2208            addtag(  # BitsPerSample
2209                258, 3, samplesperpixel, (bitspersample,) * samplesperpixel
2210            )
2211        else:
2212            addtag(258, 3, 1, bitspersample)
2213        if extrasamples:
2214            if extrasamples_ is not None:
2215                if extrasamples != len(extrasamples_):
2216                    raise ValueError('wrong number of extrasamples specified')
2217                addtag(338, 3, extrasamples, extrasamples_)
2218            elif photometric == RGB and extrasamples == 1:
2219                # Unassociated alpha channel
2220                addtag(338, 3, 1, 2)
2221            else:
2222                # Unspecified alpha channel
2223                addtag(338, 3, extrasamples, (0,) * extrasamples)
2224
2225        if jpegtables is not None:
2226            addtag(347, 7, len(jpegtables), jpegtables)
2227
2228        if (
2229            compressiontag == 7
2230            and planarconfig == 1
2231            and photometric in (RGB, YCBCR)
2232        ):
2233            # JPEG compression with subsampling
2234            # TODO: use JPEGTables for multiple tiles or strips
2235            if subsampling is None:
2236                subsampling = (2, 2)
2237            elif subsampling not in ((1, 1), (2, 1), (2, 2), (4, 1)):
2238                raise ValueError('invalid subsampling factors')
2239            maxsampling = max(subsampling) * 8
2240            if tile and (tile[-1] % maxsampling or tile[-2] % maxsampling):
2241                raise ValueError(f'tile shape not a multiple of {maxsampling}')
2242            if extrasamples > 1:
2243                raise ValueError('JPEG subsampling requires RGB(A) images')
2244            addtag(530, 3, 2, subsampling)  # YCbCrSubSampling
2245            # use PhotometricInterpretation YCBCR by default
2246            outcolorspace = enumarg(
2247                TIFF.PHOTOMETRIC, compressionargs.get('outcolorspace', 6)
2248            )
2249            compressionargs['subsampling'] = subsampling
2250            compressionargs['colorspace'] = photometric.name
2251            compressionargs['outcolorspace'] = outcolorspace.name
2252            addtag(262, 3, 1, outcolorspace)
2253            # ReferenceBlackWhite is required for YCBCR
2254            if all(et[0] != 532 for et in extratags):
2255                addtag(
2256                    532, 5, 6, (0, 1, 255, 1, 128, 1, 255, 1, 128, 1, 255, 1)
2257                )
2258        else:
2259            if subsampling not in (None, (1, 1)):
2260                log_warning('TiffWriter: cannot apply subsampling')
2261            subsampling = None
2262            maxsampling = 1
2263            addtag(262, 3, 1, photometric.value)  # PhotometricInterpretation
2264            if photometric == YCBCR:
2265                # YCbCrSubSampling and ReferenceBlackWhite
2266                addtag(530, 3, 2, (1, 1))
2267                if all(et[0] != 532 for et in extratags):
2268                    addtag(
2269                        532,
2270                        5,
2271                        6,
2272                        (0, 1, 255, 1, 128, 1, 255, 1, 128, 1, 255, 1),
2273                    )
2274        if resolution is not None:
2275            addtag(282, 5, 1, rational(resolution[0]))  # XResolution
2276            addtag(283, 5, 1, rational(resolution[1]))  # YResolution
2277            if len(resolution) > 2:
2278                unit = resolution[2]
2279                unit = 1 if unit is None else enumarg(TIFF.RESUNIT, unit)
2280            elif self._imagej:
2281                unit = 1
2282            else:
2283                unit = 2
2284            addtag(296, 3, 1, unit)  # ResolutionUnit
2285        elif not self._imagej:
2286            addtag(282, 5, 1, (1, 1))  # XResolution
2287            addtag(283, 5, 1, (1, 1))  # YResolution
2288            addtag(296, 3, 1, 1)  # ResolutionUnit
2289
2290        def bytecount_format(
2291            bytecounts, compression=compression, size=offsetsize
2292        ):
2293            # return small bytecount format
2294            if len(bytecounts) == 1:
2295                return {4: 'I', 8: 'Q'}[size]
2296            bytecount = bytecounts[0]
2297            if compression:
2298                bytecount = bytecount * 10
2299            if bytecount < 2 ** 16:
2300                return 'H'
2301            if bytecount < 2 ** 32:
2302                return 'I'
2303            if size == 4:
2304                return 'I'
2305            return 'Q'
2306
2307        # can save data array contiguous
2308        contiguous = not (compression or packints or bilevel)
2309        if tile:
2310            # one chunk per tile per plane
2311            if len(tile) == 2:
2312                tiles = (
2313                    (storedshape[3] + tile[0] - 1) // tile[0],
2314                    (storedshape[4] + tile[1] - 1) // tile[1],
2315                )
2316                contiguous = (
2317                    contiguous
2318                    and storedshape[3] == tile[0]
2319                    and storedshape[4] == tile[1]
2320                )
2321            else:
2322                tiles = (
2323                    (storedshape[2] + tile[0] - 1) // tile[0],
2324                    (storedshape[3] + tile[1] - 1) // tile[1],
2325                    (storedshape[4] + tile[2] - 1) // tile[2],
2326                )
2327                contiguous = (
2328                    contiguous
2329                    and storedshape[2] == tile[0]
2330                    and storedshape[3] == tile[1]
2331                    and storedshape[4] == tile[2]
2332                )
2333            numtiles = product(tiles) * storedshape[1]
2334            databytecounts = [
2335                product(tile) * storedshape[-1] * datadtype.itemsize
2336            ] * numtiles
2337            bytecountformat = bytecount_format(databytecounts)
2338            addtag(tagbytecounts, bytecountformat, numtiles, databytecounts)
2339            addtag(tagoffsets, offsetformat, numtiles, [0] * numtiles)
2340            bytecountformat = bytecountformat * numtiles
2341            if contiguous or dataiter is not None:
2342                pass
2343            else:
2344                dataiter = iter_tiles(data, tile, tiles)
2345
2346        elif contiguous and rowsperstrip is None:
2347            count = storedshape[1] * storedshape[2]
2348            databytecounts = [
2349                product(storedshape[3:]) * datadtype.itemsize
2350            ] * count
2351            bytecountformat = bytecount_format(databytecounts)
2352            addtag(tagbytecounts, bytecountformat, count, databytecounts)
2353            addtag(tagoffsets, offsetformat, count, [0] * count)
2354            addtag(278, 4, 1, storedshape[-3])  # RowsPerStrip
2355            bytecountformat = bytecountformat * storedshape[1]
2356            if contiguous or dataiter is not None:
2357                pass
2358            else:
2359                dataiter = iter_images(data)
2360
2361        else:
2362            # use rowsperstrip
2363            rowsize = product(storedshape[-2:]) * datadtype.itemsize
2364            if rowsperstrip is None:
2365                # compress ~64 KB chunks by default
2366                # TIFF-EP requires <= 64 KB
2367                if compression:
2368                    rowsperstrip = 65536 // rowsize
2369                else:
2370                    rowsperstrip = storedshape[-3]
2371            if rowsperstrip < 1:
2372                rowsperstrip = maxsampling
2373            elif rowsperstrip > storedshape[-3]:
2374                rowsperstrip = storedshape[-3]
2375            elif subsampling and rowsperstrip % maxsampling:
2376                rowsperstrip = (
2377                    math.ceil(rowsperstrip / maxsampling) * maxsampling
2378                )
2379            addtag(278, 4, 1, rowsperstrip)  # RowsPerStrip
2380
2381            numstrips1 = (storedshape[-3] + rowsperstrip - 1) // rowsperstrip
2382            numstrips = numstrips1 * storedshape[1] * storedshape[2]
2383            # TODO: save bilevel data with rowsperstrip
2384            stripsize = rowsperstrip * rowsize
2385            databytecounts = [stripsize] * numstrips
2386            stripsize -= rowsize * (
2387                numstrips1 * rowsperstrip - storedshape[-3]
2388            )
2389            for i in range(numstrips1 - 1, numstrips, numstrips1):
2390                databytecounts[i] = stripsize
2391            bytecountformat = bytecount_format(databytecounts)
2392            addtag(tagbytecounts, bytecountformat, numstrips, databytecounts)
2393            addtag(tagoffsets, offsetformat, numstrips, [0] * numstrips)
2394            bytecountformat = bytecountformat * numstrips
2395
2396            if contiguous or dataiter is not None:
2397                pass
2398            else:
2399                dataiter = iter_images(data)
2400
2401        if data is None and not contiguous:
2402            raise ValueError('cannot write non-contiguous empty file')
2403
2404        # add extra tags from user
2405        for t in extratags:
2406            addtag(*t)
2407
2408        # TODO: check TIFFReadDirectoryCheckOrder warning in files containing
2409        #   multiple tags of same code
2410        # the entries in an IFD must be sorted in ascending order by tag code
2411        tags = sorted(tags, key=lambda x: x[0])
2412
2413        # define compress function
2414        if bilevel:
2415            if compressiontag == 1:
2416
2417                def compress(data, level=None):
2418                    return numpy.packbits(data, axis=-2).tobytes()
2419
2420            elif compressiontag in (5, 32773):
2421                # LZW, PackBits
2422                def compress(
2423                    data,
2424                    compressor=TIFF.COMPRESSORS[compressiontag],
2425                    kwargs=compressionargs,
2426                ):
2427                    data = numpy.packbits(data, axis=-2).tobytes()
2428                    return compressor(data, **kwargs)
2429
2430            else:
2431                raise ValueError('cannot compress bilevel image')
2432
2433        elif compression:
2434            compressor = TIFF.COMPRESSORS[compressiontag]
2435
2436            if compressiontag == 32773:  # PackBits
2437                compressionargs['axis'] = -2
2438
2439            if subsampling:
2440                # JPEG with subsampling
2441                def compress(
2442                    data, compressor=compressor, kwargs=compressionargs
2443                ):
2444                    return compressor(data, **kwargs)
2445
2446            elif predictor:
2447
2448                def compress(
2449                    data,
2450                    predictor=predictor,
2451                    compressor=compressor,
2452                    kwargs=compressionargs,
2453                ):
2454                    data = predictor(data, axis=-2)
2455                    return compressor(data, **kwargs)
2456
2457            elif compressionargs:
2458
2459                def compress(
2460                    data, compressor=compressor, kwargs=compressionargs
2461                ):
2462                    return compressor(data, **kwargs)
2463
2464            else:
2465                compress = compressor
2466
2467        elif packints:
2468
2469            def compress(data, bps=bitspersample):
2470                return packints_encode(data, bps, axis=-2)
2471
2472        else:
2473            compress = False
2474
2475        del compression
2476
2477        fhpos = fh.tell()
2478        if (
2479            not (offsetsize > 4 or self._imagej or compress)
2480            and fhpos + datasize > 2 ** 32 - 1
2481        ):
2482            raise ValueError('data too large for standard TIFF file')
2483
2484        # if not compressed or multi-tiled, write the first IFD and then
2485        # all data contiguously; else, write all IFDs and data interleaved
2486        for pageindex in range(1 if contiguous else storedshape[0]):
2487
2488            ifdpos = fhpos
2489            if ifdpos % 2:
2490                # location of IFD must begin on a word boundary
2491                fh.write(b'\x00')
2492                ifdpos += 1
2493
2494            if self._subifdslevel < 0:
2495                # update pointer at ifdoffset
2496                fh.seek(self._ifdoffset)
2497                fh.write(pack(offsetformat, ifdpos))
2498
2499            fh.seek(ifdpos)
2500
2501            # create IFD in memory
2502            if pageindex < 2:
2503                subifdsoffsets = None
2504                ifd = io.BytesIO()
2505                ifd.write(pack(tagnoformat, len(tags)))
2506                tagoffset = ifd.tell()
2507                ifd.write(b''.join(t[1] for t in tags))
2508                ifdoffset = ifd.tell()
2509                ifd.write(pack(offsetformat, 0))  # offset to next IFD
2510                # write tag values and patch offsets in ifdentries
2511                for tagindex, tag in enumerate(tags):
2512                    offset = tagoffset + tagindex * tagsize + 4 + offsetsize
2513                    code = tag[0]
2514                    value = tag[2]
2515                    if value:
2516                        pos = ifd.tell()
2517                        if pos % 2:
2518                            # tag value is expected to begin on word boundary
2519                            ifd.write(b'\x00')
2520                            pos += 1
2521                        ifd.seek(offset)
2522                        ifd.write(pack(offsetformat, ifdpos + pos))
2523                        ifd.seek(pos)
2524                        ifd.write(value)
2525                        if code == tagoffsets:
2526                            dataoffsetsoffset = offset, pos
2527                        elif code == tagbytecounts:
2528                            databytecountsoffset = offset, pos
2529                        elif code == 270:
2530                            self._descriptiontag.offset = (
2531                                ifdpos + tagoffset + tagindex * tagsize
2532                            )
2533                            self._descriptiontag.valueoffset = ifdpos + pos
2534                        elif code == 330:
2535                            subifdsoffsets = offset, pos
2536                    elif code == tagoffsets:
2537                        dataoffsetsoffset = offset, None
2538                    elif code == tagbytecounts:
2539                        databytecountsoffset = offset, None
2540                    elif code == 270:
2541                        self._descriptiontag.offset = (
2542                            ifdpos + tagoffset + tagindex * tagsize
2543                        )
2544                        self._descriptiontag.valueoffset = (
2545                            self._descriptiontag.offset + offsetsize + 4
2546                        )
2547                    elif code == 330:
2548                        subifdsoffsets = offset, None
2549                ifdsize = ifd.tell()
2550                if ifdsize % 2:
2551                    ifd.write(b'\x00')
2552                    ifdsize += 1
2553
2554            # write IFD later when strip/tile bytecounts and offsets are known
2555            fh.seek(ifdsize, os.SEEK_CUR)
2556
2557            # write image data
2558            dataoffset = fh.tell()
2559            if align is None:
2560                align = 16
2561            skip = (align - (dataoffset % align)) % align
2562            fh.seek(skip, os.SEEK_CUR)
2563            dataoffset += skip
2564            if contiguous:
2565                if data is None:
2566                    fh.write_empty(datasize)
2567                elif dataiter is not None:
2568                    for pagedata in dataiter:
2569                        if pagedata.dtype != datadtype:
2570                            raise ValueError(
2571                                'dtype of iterable does not match dtype'
2572                            )
2573                        fh.write_array(pagedata.reshape(storedshape[1:]))
2574                else:
2575                    fh.write_array(data)
2576            elif tile:
2577                if storedshape[-1] == 1:
2578                    tileshape = tile
2579                else:
2580                    tileshape = tile + (storedshape[-1],)
2581                tilesize = product(tileshape) * datadtype.itemsize
2582                if data is None:
2583                    fh.write_empty(numtiles * tilesize)
2584                elif compress:
2585                    isbytes = True
2586                    for tileindex in range(storedshape[1] * product(tiles)):
2587                        chunk = next(dataiter)
2588                        if chunk is None:
2589                            databytecounts[tileindex] = 0
2590                            continue
2591                        if isbytes and isinstance(chunk, bytes):
2592                            # pre-compressed
2593                            pass
2594                        else:
2595                            if chunk.nbytes != tilesize:
2596                                chunk = pad_tile(chunk, tileshape, datadtype)
2597                            isbytes = False
2598                            chunk = compress(chunk)
2599                        fh.write(chunk)
2600                        databytecounts[tileindex] = len(chunk)
2601                else:
2602                    for tileindex in range(storedshape[1] * product(tiles)):
2603                        chunk = next(dataiter)
2604                        if chunk is None:
2605                            # databytecounts[tileindex] = 0  # not contiguous
2606                            fh.write_empty(tilesize)
2607                            continue
2608                        if chunk.nbytes != tilesize:
2609                            chunk = pad_tile(chunk, tileshape, datadtype)
2610                        fh.write_array(chunk)
2611            elif compress:
2612                # write one strip per rowsperstrip
2613                numstrips = (
2614                    storedshape[-3] + rowsperstrip - 1
2615                ) // rowsperstrip
2616                stripindex = 0
2617                pagedata = next(dataiter).reshape(storedshape[1:])
2618                if pagedata.dtype != datadtype:
2619                    raise ValueError('dtype of iterable does not match dtype')
2620                for plane in pagedata:
2621                    for depth in plane:
2622                        for i in range(numstrips):
2623                            strip = depth[
2624                                i * rowsperstrip : (i + 1) * rowsperstrip
2625                            ]
2626                            strip = compress(strip)
2627                            fh.write(strip)
2628                            databytecounts[stripindex] = len(strip)
2629                            stripindex += 1
2630            else:
2631                pagedata = next(dataiter).reshape(storedshape[1:])
2632                if pagedata.dtype != datadtype:
2633                    raise ValueError('dtype of iterable does not match dtype')
2634                fh.write_array(pagedata)
2635
2636            # update strip/tile offsets
2637            offset, pos = dataoffsetsoffset
2638            ifd.seek(offset)
2639            if pos:
2640                ifd.write(pack(offsetformat, ifdpos + pos))
2641                ifd.seek(pos)
2642                offset = dataoffset
2643                for size in databytecounts:
2644                    ifd.write(pack(offsetformat, offset))
2645                    offset += size
2646            else:
2647                ifd.write(pack(offsetformat, dataoffset))
2648
2649            if compress:
2650                # update strip/tile bytecounts
2651                offset, pos = databytecountsoffset
2652                ifd.seek(offset)
2653                if pos:
2654                    ifd.write(pack(offsetformat, ifdpos + pos))
2655                    ifd.seek(pos)
2656                ifd.write(pack(bytecountformat, *databytecounts))
2657
2658            if subifdsoffsets is not None:
2659                # update and save pointer to SubIFDs tag values if necessary
2660                offset, pos = subifdsoffsets
2661                if pos is not None:
2662                    ifd.seek(offset)
2663                    ifd.write(pack(offsetformat, ifdpos + pos))
2664                    self._subifdsoffsets.append(ifdpos + pos)
2665                else:
2666                    self._subifdsoffsets.append(ifdpos + offset)
2667
2668            fhpos = fh.tell()
2669            fh.seek(ifdpos)
2670            fh.write(ifd.getbuffer())
2671            fh.flush()
2672
2673            if self._subifdslevel < 0:
2674                self._ifdoffset = ifdpos + ifdoffset
2675            else:
2676                # update SubIFDs tag values
2677                fh.seek(
2678                    self._subifdsoffsets[self._ifdindex]
2679                    + self._subifdslevel * offsetsize
2680                )
2681                fh.write(pack(offsetformat, ifdpos))
2682
2683                # update SubIFD chain offsets
2684                if self._subifdslevel == 0:
2685                    self._nextifdoffsets.append(ifdpos + ifdoffset)
2686                else:
2687                    fh.seek(self._nextifdoffsets[self._ifdindex])
2688                    fh.write(pack(offsetformat, ifdpos))
2689                    self._nextifdoffsets[self._ifdindex] = ifdpos + ifdoffset
2690                self._ifdindex += 1
2691                self._ifdindex %= len(self._subifdsoffsets)
2692
2693            fh.seek(fhpos)
2694
2695            # remove tags that should be written only once
2696            if pageindex == 0:
2697                tags = [tag for tag in tags if not tag[-1]]
2698
2699        self._storedshape = storedshape
2700        self._datashape = (1,) + inputshape
2701        self._datadtype = datadtype
2702        self._dataoffset = dataoffset
2703        self._databytecounts = databytecounts
2704
2705        if contiguous:
2706            # write remaining IFDs/tags later
2707            self._tags = tags
2708            # return offset and size of image data
2709            if returnoffset:
2710                return dataoffset, sum(databytecounts)
2711        return None
2712
2713    def overwrite_description(self, description):
2714        """Overwrite the value of the last ImageDescription tag.
2715
2716        Can be used to write OME-XML after writing the image data.
2717        Ends a contiguous series.
2718
2719        """
2720        if self._descriptiontag is None:
2721            raise ValueError('no ImageDescription tag found')
2722        self._write_remaining_pages()
2723        self._descriptiontag.overwrite(description, erase=False)
2724        self._descriptiontag = None
2725
2726    def _write_remaining_pages(self):
2727        """Write outstanding IFDs and tags to file."""
2728        if not self._tags or self._truncate or self._datashape is None:
2729            return
2730
2731        pageno = self._storedshape[0] * self._datashape[0] - 1
2732        if pageno < 1:
2733            self._tags = None
2734            self._dataoffset = None
2735            self._databytecounts = None
2736            return
2737
2738        fh = self._fh
2739        fhpos = fh.tell()
2740        if fhpos % 2:
2741            fh.write(b'\x00')
2742            fhpos += 1
2743
2744        pack = struct.pack
2745        offsetformat = self.tiff.offsetformat
2746        offsetsize = self.tiff.offsetsize
2747        tagnoformat = self.tiff.tagnoformat
2748        tagsize = self.tiff.tagsize
2749        dataoffset = self._dataoffset
2750        pagedatasize = sum(self._databytecounts)
2751        subifdsoffsets = None
2752
2753        # construct template IFD in memory
2754        # must patch offsets to next IFD and data before writing to file
2755        ifd = io.BytesIO()
2756        ifd.write(pack(tagnoformat, len(self._tags)))
2757        tagoffset = ifd.tell()
2758        ifd.write(b''.join(t[1] for t in self._tags))
2759        ifdoffset = ifd.tell()
2760        ifd.write(pack(offsetformat, 0))  # offset to next IFD
2761        # tag values
2762        for tagindex, tag in enumerate(self._tags):
2763            offset = tagoffset + tagindex * tagsize + offsetsize + 4
2764            code = tag[0]
2765            value = tag[2]
2766            if value:
2767                pos = ifd.tell()
2768                if pos % 2:
2769                    # tag value is expected to begin on word boundary
2770                    ifd.write(b'\x00')
2771                    pos += 1
2772                ifd.seek(offset)
2773                try:
2774                    ifd.write(pack(offsetformat, fhpos + pos))
2775                except Exception:  # struct.error
2776                    if self._imagej:
2777                        warnings.warn(
2778                            'TiffWriter: truncating ImageJ file', UserWarning
2779                        )
2780                        self._truncate = True
2781                        return
2782                    raise ValueError('data too large for non-BigTIFF file')
2783                ifd.seek(pos)
2784                ifd.write(value)
2785                if code == self._dataoffsetstag:
2786                    # save strip/tile offsets for later updates
2787                    dataoffsetsoffset = offset, pos
2788                elif code == 330:
2789                    # save subifds offsets for later updates
2790                    subifdsoffsets = offset, pos
2791            elif code == self._dataoffsetstag:
2792                dataoffsetsoffset = offset, None
2793            elif code == 330:
2794                subifdsoffsets = offset, None
2795
2796        ifdsize = ifd.tell()
2797        if ifdsize % 2:
2798            ifd.write(b'\x00')
2799            ifdsize += 1
2800
2801        # check if all IFDs fit in file
2802        if offsetsize < 8 and fhpos + ifdsize * pageno > 2 ** 32 - 32:
2803            if self._imagej:
2804                warnings.warn(
2805                    'TiffWriter: truncating ImageJ file', UserWarning
2806                )
2807                self._truncate = True
2808                return
2809            raise ValueError('data too large for non-BigTIFF file')
2810
2811        # assemble IFD chain in memory from IFD template
2812        ifds = io.BytesIO(bytes(ifdsize * pageno))
2813        ifdpos = fhpos
2814        for _ in range(pageno):
2815            # update strip/tile offsets in IFD
2816            dataoffset += pagedatasize  # offset to image data
2817            offset, pos = dataoffsetsoffset
2818            ifd.seek(offset)
2819            if pos is not None:
2820                ifd.write(pack(offsetformat, ifdpos + pos))
2821                ifd.seek(pos)
2822                offset = dataoffset
2823                for size in self._databytecounts:
2824                    ifd.write(pack(offsetformat, offset))
2825                    offset += size
2826            else:
2827                ifd.write(pack(offsetformat, dataoffset))
2828
2829            if subifdsoffsets is not None:
2830                offset, pos = subifdsoffsets
2831                self._subifdsoffsets.append(
2832                    ifdpos + (pos if pos is not None else offset)
2833                )
2834
2835            if self._subifdslevel < 0:
2836                if subifdsoffsets is not None:
2837                    # update pointer to SubIFDs tag values if necessary
2838                    offset, pos = subifdsoffsets
2839                    if pos is not None:
2840                        ifd.seek(offset)
2841                        ifd.write(pack(offsetformat, ifdpos + pos))
2842
2843                # update pointer at ifdoffset to point to next IFD in file
2844                ifdpos += ifdsize
2845                ifd.seek(ifdoffset)
2846                ifd.write(pack(offsetformat, ifdpos))
2847
2848            else:
2849                # update SubIFDs tag values in file
2850                fh.seek(
2851                    self._subifdsoffsets[self._ifdindex]
2852                    + self._subifdslevel * offsetsize
2853                )
2854                fh.write(pack(offsetformat, ifdpos))
2855
2856                # update SubIFD chain
2857                if self._subifdslevel == 0:
2858                    self._nextifdoffsets.append(ifdpos + ifdoffset)
2859                else:
2860                    fh.seek(self._nextifdoffsets[self._ifdindex])
2861                    fh.write(pack(offsetformat, ifdpos))
2862                    self._nextifdoffsets[self._ifdindex] = ifdpos + ifdoffset
2863                self._ifdindex += 1
2864                self._ifdindex %= len(self._subifdsoffsets)
2865                ifdpos += ifdsize
2866
2867            # write IFD entry
2868            ifds.write(ifd.getbuffer())
2869
2870        # terminate IFD chain
2871        ifdoffset += ifdsize * (pageno - 1)
2872        ifds.seek(ifdoffset)
2873        ifds.write(pack(offsetformat, 0))
2874        # write IFD chain to file
2875        fh.seek(fhpos)
2876        fh.write(ifds.getbuffer())
2877
2878        if self._subifdslevel < 0:
2879            # update file to point to new IFD chain
2880            pos = fh.tell()
2881            fh.seek(self._ifdoffset)
2882            fh.write(pack(offsetformat, fhpos))
2883            fh.flush()
2884            fh.seek(pos)
2885            self._ifdoffset = fhpos + ifdoffset
2886
2887        self._tags = None
2888        self._dataoffset = None
2889        self._databytecounts = None
2890        # do not reset _storedshape, _datashape, _datadtype
2891
2892    def _write_image_description(self):
2893        """Write metadata to ImageDescription tag."""
2894        if self._datashape is None or self._descriptiontag is None:
2895            self._descriptiontag = None
2896            return
2897
2898        if self._ome:
2899            if self._subifdslevel < 0:
2900                self._ome.addimage(
2901                    self._datadtype,
2902                    self._datashape[0 if self._datashape[0] != 1 else 1 :],
2903                    self._storedshape,
2904                    **self._metadata,
2905                )
2906            description = self._ome.tostring(declaration=True).encode()
2907        elif self._datashape[0] == 1:
2908            # description already up-to-date
2909            self._descriptiontag = None
2910            return
2911        # elif self._subifdslevel >= 0:
2912        #     # don't write metadata to SubIFDs
2913        #     return
2914        elif self._imagej:
2915            colormapped = self._colormap is not None
2916            isrgb = self._storedshape[-1] in (3, 4)
2917            description = imagej_description(
2918                self._datashape, isrgb, colormapped, **self._metadata
2919            )
2920        else:
2921            description = json_description(self._datashape, **self._metadata)
2922
2923        self._descriptiontag.overwrite(description, erase=False)
2924        self._descriptiontag = None
2925
2926    def _now(self):
2927        """Return current date and time."""
2928        return datetime.datetime.now()
2929
2930    def close(self):
2931        """Write remaining pages and close file handle."""
2932        if not self._truncate:
2933            self._write_remaining_pages()
2934        self._write_image_description()
2935        self._fh.close()
2936
2937    def __enter__(self):
2938        return self
2939
2940    def __exit__(self, exc_type, exc_value, traceback):
2941        self.close()
2942
2943
2944class TiffFile:
2945    """Read image and metadata from TIFF file.
2946
2947    TiffFile instances must be closed using the 'close' method, which is
2948    automatically called when using the 'with' context manager.
2949
2950    TiffFile instances are not thread-safe.
2951
2952    Attributes
2953    ----------
2954    pages : TiffPages
2955        Sequence of TIFF pages in file.
2956    series : list of TiffPageSeries
2957        Sequences of closely related TIFF pages. These are computed
2958        from OME, LSM, ImageJ, etc. metadata or based on similarity
2959        of page properties such as shape, dtype, and compression.
2960    is_flag : bool
2961        If True, file is of a certain format.
2962        Flags are: bigtiff, uniform, shaped, ome, imagej, stk, lsm, fluoview,
2963        nih, vista, micromanager, metaseries, mdgel, mediacy, tvips, fei,
2964        sem, scn, svs, scanimage, andor, epics, ndpi, pilatus, qpi.
2965
2966    All attributes are read-only.
2967
2968    """
2969
2970    def __init__(
2971        self,
2972        arg,
2973        mode=None,
2974        name=None,
2975        offset=None,
2976        size=None,
2977        _multifile=True,
2978        _useframes=None,
2979        _parent=None,
2980        **kwargs,
2981    ):
2982        """Initialize instance from file.
2983
2984        Parameters
2985        ----------
2986        arg : str or open file
2987            Name of file or open file object.
2988            The file objects are closed in TiffFile.close().
2989        mode : str
2990            File open mode in case 'arg' is a file name. Must be 'rb' or 'r+b'.
2991            Default is 'rb'.
2992        name : str
2993            Optional name of file in case 'arg' is a file handle.
2994        offset : int
2995            Optional start position of embedded file. By default, this is
2996            the current file position.
2997        size : int
2998            Optional size of embedded file. By default, this is the number
2999            of bytes from the 'offset' to the end of the file.
3000        kwargs : bool
3001            'is_ome': If False, disable processing of OME-XML metadata.
3002
3003        """
3004        if kwargs:
3005            # TODO: remove; formally deprecated in 2020.10.1
3006            for key in ('fastij', 'movie', 'multifile', 'multifile_close'):
3007                if key in kwargs:
3008                    del kwargs[key]
3009                    warnings.warn(
3010                        f'TiffFile: the {key!r} argument is ignored',
3011                        DeprecationWarning,
3012                        stacklevel=2,
3013                    )
3014            if 'pages' in kwargs:
3015                raise TypeError(
3016                    "the TiffFile 'pages' parameter is no longer supported."
3017                    "\n\nUse TiffFile.asarray(key=[...]) to read image data "
3018                    "from specific pages.\n"
3019                )
3020
3021            for key, value in kwargs.items():
3022                if key[:3] == 'is_' and key[3:] in TIFF.FILE_FLAGS:
3023                    if value is not None:
3024                        setattr(self, key, bool(value))
3025                else:
3026                    raise TypeError(f'unexpected keyword argument: {key}')
3027
3028        if mode not in (None, 'rb', 'r+b'):
3029            raise ValueError(f'invalid mode {mode!r}')
3030
3031        fh = FileHandle(arg, mode=mode, name=name, offset=offset, size=size)
3032        self._fh = fh
3033        self._multifile = bool(_multifile)
3034        self._files = {fh.name: self}  # cache of TiffFile instances
3035        self._decoders = {}  # cache of TiffPage.decode functions
3036        self._parent = self if _parent is None else _parent  # OME master file
3037
3038        try:
3039            fh.seek(0)
3040            header = fh.read(4)
3041            try:
3042                byteorder = {b'II': '<', b'MM': '>', b'EP': '<'}[header[:2]]
3043            except KeyError:
3044                raise TiffFileError('not a TIFF file')
3045
3046            version = struct.unpack(byteorder + 'H', header[2:4])[0]
3047            if version == 43:
3048                # BigTiff
3049                offsetsize, zero = struct.unpack(byteorder + 'HH', fh.read(4))
3050                if zero != 0 or offsetsize != 8:
3051                    raise TiffFileError('invalid BigTIFF file')
3052                if byteorder == '>':
3053                    self.tiff = TIFF.BIG_BE
3054                else:
3055                    self.tiff = TIFF.BIG_LE
3056            elif version == 42:
3057                # Classic TIFF
3058                if byteorder == '>':
3059                    self.tiff = TIFF.CLASSIC_BE
3060                elif kwargs.get('is_ndpi', False) or fh.name.endswith('ndpi'):
3061                    # NDPI uses 64 bit IFD offsets
3062                    self.tiff = TIFF.NDPI_LE
3063                else:
3064                    self.tiff = TIFF.CLASSIC_LE
3065            elif version == 0x55 or version == 0x4F52 or version == 0x5352:
3066                # Panasonic or Olympus RAW
3067                log_warning(f'RAW format 0x{version:04X} not supported')
3068                if byteorder == '>':
3069                    self.tiff = TIFF.CLASSIC_BE
3070                else:
3071                    self.tiff = TIFF.CLASSIC_LE
3072            else:
3073                raise TiffFileError(f'invalid TIFF version {version}')
3074
3075            # file handle is at offset to offset to first page
3076            self.pages = TiffPages(self)
3077
3078            if self.is_lsm and (
3079                self.filehandle.size >= 2 ** 32
3080                or self.pages[0].compression != 1
3081                or self.pages[1].compression != 1
3082            ):
3083                self._lsm_load_pages()
3084
3085            elif self.is_scanimage and not self.is_bigtiff:
3086                # ScanImage <= 2015
3087                try:
3088                    self.pages._load_virtual_frames()
3089                except Exception as exc:
3090                    log_warning(
3091                        f'load_virtual_frames: {exc.__class__.__name__}: {exc}'
3092                    )
3093
3094            elif self.is_philips:
3095                try:
3096                    self._philips_load_pages()
3097                except Exception as exc:
3098                    log_warning(
3099                        f'philips_load_pages: {exc.__class__.__name__}: {exc}'
3100                    )
3101
3102            elif self.is_ndpi:
3103                try:
3104                    self._ndpi_load_pages()
3105                except Exception as exc:
3106                    log_warning(
3107                        f'_ndpi_load_pages: {exc.__class__.__name__}: {exc}'
3108                    )
3109
3110            elif _useframes:
3111                self.pages.useframes = True
3112
3113        except Exception:
3114            fh.close()
3115            raise
3116
3117    @property
3118    def byteorder(self):
3119        return self.tiff.byteorder
3120
3121    @property
3122    def filehandle(self):
3123        """Return file handle."""
3124        return self._fh
3125
3126    @property
3127    def filename(self):
3128        """Return name of file handle."""
3129        return self._fh.name
3130
3131    @lazyattr
3132    def fstat(self):
3133        """Return status of file handle as stat_result object."""
3134        try:
3135            return os.fstat(self._fh.fileno())
3136        except Exception:  # io.UnsupportedOperation
3137            return None
3138
3139    def close(self):
3140        """Close open file handle(s)."""
3141        for tif in self._files.values():
3142            tif.filehandle.close()
3143
3144    def asarray(
3145        self,
3146        key=None,
3147        series=None,
3148        level=None,
3149        squeeze=None,
3150        out=None,
3151        maxworkers=None,
3152    ):
3153        """Return image data from selected TIFF page(s) as numpy array.
3154
3155        By default, the data from the first series is returned.
3156
3157        Parameters
3158        ----------
3159        key : int, slice, or sequence of indices
3160            Defines which pages to return as array.
3161            If None (default), data from a series (default 0) is returned.
3162            If not None, data from the specified pages in the whole file
3163            (if 'series' is None) or a specified series are returned as a
3164            stacked array.
3165            Requesting an array from multiple pages that are not compatible
3166            wrt. shape, dtype, compression etc. is undefined, i.e. may crash
3167            or return incorrect values.
3168        series : int or TiffPageSeries
3169            Defines which series of pages to return as array.
3170        level : int
3171            Defines which pyramid level of a series to return as array.
3172        squeeze : bool
3173            If True, all length-1 dimensions (except X and Y) are squeezed
3174            out from the array.
3175            If False, single pages are returned as 5D array (TiffPage.shaped).
3176            For series, the shape of the returned array also includes singlet
3177            dimensions specified in some file formats. E.g. ImageJ series, and
3178            most commonly also OME series, are returned in TZCYXS order.
3179            If None (default), all but "shaped" series are squeezed.
3180        out : numpy.ndarray, str, or file-like object
3181            Buffer where image data are saved.
3182            If None (default), a new array is created.
3183            If numpy.ndarray, a writable array of compatible dtype and shape.
3184            If 'memmap', directly memory-map the image data in the TIFF file
3185            if possible; else create a memory-mapped array in a temporary file.
3186            If str or open file, the file name or file object used to
3187            create a memory-map to an array stored in a binary file on disk.
3188        maxworkers : int or None
3189            Maximum number of threads to concurrently get data from multiple
3190            pages or compressed segments.
3191            If None (default), up to half the CPU cores are used.
3192            If 1, multi-threading is disabled.
3193            Reading data from file is limited to a single thread.
3194            Using multiple threads can significantly speed up this function
3195            if the bottleneck is decoding compressed data, e.g. in case of
3196            large LZW compressed LSM files or JPEG compressed tiled slides.
3197            If the bottleneck is I/O or pure Python code, using multiple
3198            threads might be detrimental.
3199
3200        Returns
3201        -------
3202        numpy.ndarray
3203            Image data from the specified pages.
3204            See TiffPage.asarray for operations that are applied (or not)
3205            to the raw data stored in the file.
3206
3207        """
3208        if not self.pages:
3209            return numpy.array([])
3210        if key is None and series is None:
3211            series = 0
3212        if series is None:
3213            pages = self.pages
3214        else:
3215            try:
3216                series = self.series[series]
3217            except (KeyError, TypeError):
3218                pass
3219            if level is not None:
3220                series = series.levels[level]
3221            pages = series.pages
3222
3223        if key is None:
3224            pass
3225        elif series is None:
3226            pages = self.pages._getlist(key)
3227        elif isinstance(key, (int, numpy.integer)):
3228            pages = [pages[key]]
3229        elif isinstance(key, slice):
3230            pages = pages[key]
3231        elif isinstance(key, Iterable):
3232            pages = [pages[k] for k in key]
3233        else:
3234            raise TypeError('key must be an int, slice, or sequence')
3235
3236        if not pages:
3237            raise ValueError('no pages selected')
3238
3239        if key is None and series and series.offset:
3240            typecode = self.byteorder + series.dtype.char
3241            if (
3242                pages[0].is_memmappable
3243                and isinstance(out, str)
3244                and out == 'memmap'
3245            ):
3246                # direct mapping
3247                shape = series.get_shape(squeeze)
3248                result = self.filehandle.memmap_array(
3249                    typecode, shape, series.offset
3250                )
3251            else:
3252                # read into output
3253                shape = series.get_shape(squeeze)
3254                if out is not None:
3255                    out = create_output(out, shape, series.dtype)
3256                self.filehandle.seek(series.offset)
3257                result = self.filehandle.read_array(
3258                    typecode, product(shape), out=out
3259                )
3260        elif len(pages) == 1:
3261            result = pages[0].asarray(out=out, maxworkers=maxworkers)
3262        else:
3263            result = stack_pages(pages, out=out, maxworkers=maxworkers)
3264
3265        if result is None:
3266            return None
3267
3268        if key is None:
3269            shape = series.get_shape(squeeze)
3270            try:
3271                result.shape = shape
3272            except ValueError:
3273                try:
3274                    log_warning(
3275                        'TiffFile.asarray: '
3276                        f'failed to reshape {result.shape} to {shape}'
3277                    )
3278                    # try series of expected shapes
3279                    result.shape = (-1,) + shape
3280                except ValueError:
3281                    # revert to generic shape
3282                    result.shape = (-1,) + pages[0].shape
3283        elif len(pages) == 1:
3284            if squeeze is None:
3285                squeeze = True
3286            result.shape = pages[0].shape if squeeze else pages[0].shaped
3287        else:
3288            if squeeze is None:
3289                squeeze = True
3290            result.shape = (-1,) + (
3291                pages[0].shape if squeeze else pages[0].shaped
3292            )
3293        return result
3294
3295    def aszarr(self, key=None, series=None, level=None, **kwargs):
3296        """Return image data from selected TIFF page(s) as zarr storage."""
3297        if not self.pages:
3298            raise NotImplementedError('empty zarr arrays not supported')
3299        if key is None and series is None:
3300            return self.series[0].aszarr(level=level, **kwargs)
3301        if series is None:
3302            pages = self.pages
3303        else:
3304            try:
3305                series = self.series[series]
3306            except (KeyError, TypeError):
3307                pass
3308            if key is None:
3309                return series.aszarr(level=level, **kwargs)
3310            pages = series.pages
3311        if isinstance(key, (int, numpy.integer)):
3312            return pages[key].aszarr(**kwargs)
3313        raise TypeError('key must be an integer index')
3314
3315    @lazyattr
3316    def series(self):
3317        """Return related pages as TiffPageSeries.
3318
3319        Side effect: after calling this function, TiffFile.pages might contain
3320        TiffPage and TiffFrame instances.
3321
3322        """
3323        if not self.pages:
3324            return []
3325
3326        useframes = self.pages.useframes
3327        keyframe = self.pages.keyframe.index
3328        series = []
3329        for name in (
3330            'shaped',
3331            'lsm',
3332            'ome',
3333            'imagej',
3334            'fluoview',
3335            'stk',
3336            'sis',
3337            'svs',
3338            'scn',
3339            'qpi',
3340            'ndpi',
3341            'bif',
3342            'scanimage',
3343            'mdgel',  # adds second page to cache
3344            'uniform',
3345        ):
3346            if getattr(self, 'is_' + name, False):
3347                series = getattr(self, '_series_' + name)()
3348                if not series and name == 'ome' and self.is_imagej:
3349                    # try ImageJ series if OME series fails.
3350                    # clear pages cache since _series_ome() might leave some
3351                    # frames without keyframe
3352                    self.pages._clear()
3353                    continue
3354                break
3355        self.pages.useframes = useframes
3356        self.pages.keyframe = keyframe
3357        if not series:
3358            series = self._series_generic()
3359
3360        # remove empty series, e.g. in MD Gel files
3361        # series = [s for s in series if product(s.shape) > 0]
3362
3363        for i, s in enumerate(series):
3364            s.index = i
3365        return series
3366
3367    def _series_uniform(self):
3368        """Return all images in file as single series."""
3369        page = self.pages[0]
3370        validate = not (page.is_scanimage or page.is_nih)
3371        pages = self.pages._getlist(validate=validate)
3372        shape = (len(pages),) + page.shape
3373        axes = 'I' + page.axes
3374        dtype = page.dtype
3375        if page.is_nih:
3376            kind = 'NIHImage'
3377        else:
3378            kind = 'Uniform'
3379        return [TiffPageSeries(pages, shape, dtype, axes, kind=kind)]
3380
3381    def _series_generic(self):
3382        """Return image series in file.
3383
3384        A series is a sequence of TiffPages with the same hash.
3385
3386        """
3387        pages = self.pages
3388        pages._clear(False)
3389        pages.useframes = False
3390        if pages.cache:
3391            pages._load()
3392
3393        series = []
3394        keys = []
3395        seriesdict = {}
3396
3397        def addpage(page):
3398            # add page to seriesdict
3399            if not page.shape:  # or product(page.shape) == 0:
3400                return
3401            key = page.hash
3402            if key in seriesdict:
3403                for p in seriesdict[key]:
3404                    if p.offset == page.offset:
3405                        break  # remove duplicate page
3406                else:
3407                    seriesdict[key].append(page)
3408            else:
3409                keys.append(key)
3410                seriesdict[key] = [page]
3411
3412        for page in pages:
3413            addpage(page)
3414            if page.subifds is not None:
3415                for i, offset in enumerate(page.subifds):
3416                    if offset < 8:
3417                        continue
3418                    try:
3419                        self._fh.seek(offset)
3420                        subifd = TiffPage(self, (page.index, i))
3421                    except Exception as exc:
3422                        log_warning(
3423                            f'Generic series: {exc.__class__.__name__}: {exc}'
3424                        )
3425                    else:
3426                        addpage(subifd)
3427
3428        for key in keys:
3429            pages = seriesdict[key]
3430            page = pages[0]
3431            shape = (len(pages),) + page.shape
3432            axes = 'I' + page.axes
3433            if 'S' not in axes:
3434                shape += (1,)
3435                axes += 'S'
3436            series.append(
3437                TiffPageSeries(pages, shape, page.dtype, axes, kind='Generic')
3438            )
3439
3440        self.is_uniform = len(series) == 1
3441        pyramidize_series(series)
3442        return series
3443
3444    def _series_shaped(self):
3445        """Return image series in "shaped" file."""
3446
3447        def append(series, pages, axes, shape, reshape, name, truncated):
3448            # append TiffPageSeries to series
3449            page = pages[0]
3450            if not axes:
3451                shape = page.shape
3452                axes = page.axes
3453                if len(pages) > 1:
3454                    shape = (len(pages),) + shape
3455                    axes = 'Q' + axes
3456            size = product(shape)
3457            resize = product(reshape)
3458            if page.is_contiguous and resize > size and resize % size == 0:
3459                if truncated is None:
3460                    truncated = True
3461                axes = 'Q' + axes
3462                shape = (resize // size,) + shape
3463            try:
3464                axes = reshape_axes(axes, shape, reshape)
3465                shape = reshape
3466            except ValueError as exc:
3467                log_warning(f'Shaped series: {exc.__class__.__name__}: {exc}')
3468            series.append(
3469                TiffPageSeries(
3470                    pages,
3471                    shape,
3472                    page.dtype,
3473                    axes,
3474                    name=name,
3475                    kind='Shaped',
3476                    truncated=truncated,
3477                    squeeze=False,
3478                )
3479            )
3480
3481        def detect_series(pages, series, issubifds=False):
3482            lenpages = len(pages)
3483            keyframe = axes = shape = reshape = name = None
3484            index = 0
3485            while True:
3486                if index >= lenpages:
3487                    break
3488                if issubifds:
3489                    keyframe = pages[0]
3490                else:
3491                    # new keyframe; start of new series
3492                    pages.keyframe = index
3493                    keyframe = pages.keyframe
3494                if not keyframe.is_shaped:
3495                    log_warning(
3496                        'Shaped series: invalid metadata or corrupted file'
3497                    )
3498                    return None
3499                # read metadata
3500                axes = None
3501                shape = None
3502                metadata = json_description_metadata(keyframe.is_shaped)
3503                name = metadata.get('name', '')
3504                reshape = metadata['shape']
3505                truncated = None if keyframe.subifds is None else False
3506                truncated = metadata.get('truncated', truncated)
3507                if 'axes' in metadata:
3508                    axes = metadata['axes']
3509                    if len(axes) == len(reshape):
3510                        shape = reshape
3511                    else:
3512                        axes = ''
3513                        log_warning('Shaped series: axes do not match shape')
3514                # skip pages if possible
3515                spages = [keyframe]
3516                size = product(reshape)
3517                if size > 0:
3518                    npages, mod = divmod(size, product(keyframe.shape))
3519                else:
3520                    npages = 1
3521                    mod = 0
3522                if mod:
3523                    log_warning(
3524                        'Shaped series: series shape does not match page shape'
3525                    )
3526                    return None
3527                if 1 < npages <= lenpages - index:
3528                    size *= keyframe._dtype.itemsize
3529                    if truncated:
3530                        npages = 1
3531                    elif (
3532                        keyframe.is_final
3533                        and keyframe.offset + size < pages[index + 1].offset
3534                        and keyframe.subifds is None
3535                    ):
3536                        truncated = False
3537                    else:
3538                        # must read all pages for series
3539                        truncated = False
3540                        for j in range(index + 1, index + npages):
3541                            page = pages[j]
3542                            page.keyframe = keyframe
3543                            spages.append(page)
3544                append(series, spages, axes, shape, reshape, name, truncated)
3545                index += npages
3546
3547                # create series from SubIFDs
3548                if keyframe.subifds:
3549                    for i, offset in enumerate(keyframe.subifds):
3550                        if offset < 8:
3551                            continue
3552                        subifds = []
3553                        for j, page in enumerate(spages):
3554                            # if page.subifds is not None:
3555                            try:
3556                                self._fh.seek(page.subifds[i])
3557                                if j == 0:
3558                                    subifd = TiffPage(self, (page.index, i))
3559                                    keysubifd = subifd
3560                                else:
3561                                    subifd = TiffFrame(
3562                                        self,
3563                                        (page.index, i),
3564                                        keyframe=keysubifd,
3565                                    )
3566                            except Exception as exc:
3567                                log_warning(
3568                                    f'Generic series: {exc.__class__.__name__}'
3569                                    f': {exc}'
3570                                )
3571                                return None
3572                            subifds.append(subifd)
3573                        if subifds:
3574                            series = detect_series(subifds, series, True)
3575                            if series is None:
3576                                return None
3577            return series
3578
3579        self.pages.useframes = True
3580        series = detect_series(self.pages, [])
3581        if series is None:
3582            return None
3583        self.is_uniform = len(series) == 1
3584        pyramidize_series(series, isreduced=True)
3585        return series
3586
3587    def _series_imagej(self):
3588        """Return image series in ImageJ file."""
3589        # ImageJ's dimension order is TZCYXS
3590        # TODO: fix loading of color, composite, or palette images
3591        pages = self.pages
3592        pages.useframes = True
3593        pages.keyframe = 0
3594        page = pages[0]
3595        meta = self.imagej_metadata
3596
3597        def is_virtual():
3598            # ImageJ virtual hyperstacks store all image metadata in the first
3599            # page and image data are stored contiguously before the second
3600            # page, if any
3601            if not page.is_final:
3602                return False
3603            images = meta.get('images', 0)
3604            if images <= 1:
3605                return False
3606            offset, count = page.is_contiguous
3607            if (
3608                count != product(page.shape) * page.bitspersample // 8
3609                or offset + count * images > self.filehandle.size
3610            ):
3611                raise ValueError
3612            # check that next page is stored after data
3613            if len(pages) > 1 and offset + count * images > pages[1].offset:
3614                return False
3615            return True
3616
3617        try:
3618            isvirtual = is_virtual()
3619        except (ValueError, RuntimeError) as exc:
3620            log_warning(
3621                f'ImageJ series: invalid metadata or corrupted file ({exc})'
3622            )
3623            return None
3624        if isvirtual:
3625            # no need to read other pages
3626            pages = [page]
3627        else:
3628            pages = pages[:]
3629
3630        images = meta.get('images', len(pages))
3631        frames = meta.get('frames', 1)
3632        slices = meta.get('slices', 1)
3633        channels = meta.get('channels', 1)
3634
3635        shape = (frames, slices, channels)
3636        axes = 'TZC'
3637
3638        remain = images // product(shape)
3639        if remain > 1:
3640            log_warning("ImageJ series: detected extra dimension 'I'")
3641            shape = (remain,) + shape
3642            axes = 'I' + axes
3643
3644        if page.shaped[0] > 1:
3645            # planar storage, S == C, saved by Bio-Formats
3646            if page.shaped[0] != channels:
3647                log_warning(
3648                    f'ImageJ series: number of channels {channels} '
3649                    f'do not match separate samples {page.shaped[0]}'
3650                )
3651            shape = shape[:-1] + page.shape
3652            axes += page.axes[1:]
3653        elif page.shaped[-1] == channels and channels > 1:
3654            # keep contig storage, C = 1
3655            shape = (frames, slices, 1) + page.shape
3656            axes += page.axes
3657        else:
3658            shape += page.shape
3659            axes += page.axes
3660
3661        if 'S' not in axes:
3662            shape += (1,)
3663            axes += 'S'
3664        # assert axes.endswith('TZCYXS'), axes
3665
3666        truncated = (
3667            isvirtual
3668            and len(self.pages) == 1
3669            and page.is_contiguous[1]
3670            != (product(shape) * page.bitspersample // 8)
3671        )
3672
3673        self.is_uniform = True
3674
3675        return [
3676            TiffPageSeries(
3677                pages,
3678                shape,
3679                page.dtype,
3680                axes,
3681                kind='ImageJ',
3682                truncated=truncated,
3683            )
3684        ]
3685
3686    def _series_scanimage(self):
3687        """Return image series in ScanImage file."""
3688        pages = self.pages._getlist(validate=False)
3689        page = pages[0]
3690        dtype = page.dtype
3691        shape = None
3692
3693        framedata = self.scanimage_metadata.get('FrameData', {})
3694        if 'SI.hChannels.channelSave' in framedata:
3695            try:
3696                channels = framedata['SI.hChannels.channelSave']
3697                try:
3698                    # channelSave is a list
3699                    channels = len(channels)
3700                except TypeError:
3701                    # channelSave is an int
3702                    channels = int(channels)
3703                # slices = framedata.get(
3704                #    'SI.hStackManager.actualNumSlices',
3705                #     framedata.get('SI.hStackManager.numSlices', None),
3706                # )
3707                # if slices is None:
3708                #     raise ValueError('unable to determine numSlices')
3709                slices = None
3710                try:
3711                    frames = int(framedata['SI.hStackManager.framesPerSlice'])
3712                except Exception:
3713                    # framesPerSlice is inf
3714                    slices = 1
3715                    if len(pages) % channels:
3716                        raise ValueError('unable to determine framesPerSlice')
3717                    frames = len(pages) // channels
3718                if slices is None:
3719                    slices = max(len(pages) // (frames * channels), 1)
3720                shape = (slices, frames, channels) + page.shape
3721                axes = 'ZTC' + page.axes
3722            except Exception as exc:
3723                log_warning(f'ScanImage series: {exc}')
3724
3725        # TODO: older versions of ScanImage store non-varying frame data in
3726        # the ImageDescription tag. Candidates are scanimage.SI5.channelsSave,
3727        # scanimage.SI5.stackNumSlices, scanimage.SI5.acqNumFrames
3728        # scanimage.SI4., state.acq.numberOfFrames, state.acq.numberOfFrames...
3729
3730        if shape is None:
3731            shape = (len(pages),) + page.shape
3732            axes = 'I' + page.axes
3733
3734        return [TiffPageSeries(pages, shape, dtype, axes, kind='ScanImage')]
3735
3736    def _series_fluoview(self):
3737        """Return image series in FluoView file."""
3738        pages = self.pages._getlist(validate=False)
3739
3740        mm = self.fluoview_metadata
3741        mmhd = list(reversed(mm['Dimensions']))
3742        axes = ''.join(TIFF.MM_DIMENSIONS.get(i[0].upper(), 'Q') for i in mmhd)
3743        shape = tuple(int(i[1]) for i in mmhd)
3744        self.is_uniform = True
3745        return [
3746            TiffPageSeries(
3747                pages,
3748                shape,
3749                pages[0].dtype,
3750                axes,
3751                name=mm['ImageName'],
3752                kind='FluoView',
3753            )
3754        ]
3755
3756    def _series_mdgel(self):
3757        """Return image series in MD Gel file."""
3758        # only a single page, scaled according to metadata in second page
3759        self.pages.useframes = False
3760        self.pages.keyframe = 0
3761        md = self.mdgel_metadata
3762        if md['FileTag'] in (2, 128):
3763            dtype = numpy.dtype('float32')
3764            scale = md['ScalePixel']
3765            scale = scale[0] / scale[1]  # rational
3766            if md['FileTag'] == 2:
3767                # squary root data format
3768                def transform(a):
3769                    return a.astype('float32') ** 2 * scale
3770
3771            else:
3772
3773                def transform(a):
3774                    return a.astype('float32') * scale
3775
3776        else:
3777            transform = None
3778        page = self.pages[0]
3779        self.is_uniform = False
3780        return [
3781            TiffPageSeries(
3782                [page],
3783                page.shape,
3784                dtype,
3785                page.axes,
3786                transform=transform,
3787                kind='MDGel',
3788            )
3789        ]
3790
3791    def _series_ndpi(self):
3792        """Return pyramidal image series in NDPI file."""
3793        series = self._series_generic()
3794        for s in series:
3795            s.kind = 'NDPI'
3796            if s.axes[0] == 'I':
3797                s.axes = 'Z' + s.axes[1:]
3798            if s.is_pyramidal:
3799                name = s.pages[0].tags.get(65427, None)
3800                s.name = 'Baseline' if name is None else name.value
3801                continue
3802            mag = s.pages[0].tags.get(65421, None)
3803            if mag is not None:
3804                mag = mag.value
3805                if mag == -1.0:
3806                    s.name = 'Macro'
3807                elif mag == -2.0:
3808                    s.name = 'Map'
3809        return series
3810
3811    def _series_sis(self):
3812        """Return image series in Olympus SIS file."""
3813        pages = self.pages._getlist(validate=False)
3814        page = pages[0]
3815        lenpages = len(pages)
3816        md = self.sis_metadata
3817        if 'shape' in md and 'axes' in md:
3818            shape = md['shape'] + page.shape
3819            axes = md['axes'] + page.axes
3820        else:
3821            shape = (lenpages,) + page.shape
3822            axes = 'I' + page.axes
3823        self.is_uniform = True
3824        return [TiffPageSeries(pages, shape, page.dtype, axes, kind='SIS')]
3825
3826    def _series_qpi(self):
3827        """Return image series in PerkinElmer QPI file."""
3828        series = []
3829        pages = self.pages
3830        pages.cache = True
3831        pages.useframes = False
3832        pages.keyframe = 0
3833        pages._load()
3834
3835        # Baseline
3836        # TODO: get name from ImageDescription XML
3837        ifds = []
3838        index = 0
3839        axes = 'C' + pages[0].axes
3840        dtype = pages[0].dtype
3841        pshape = pages[0].shape
3842        while index < len(pages):
3843            page = pages[index]
3844            if page.shape != pshape:
3845                break
3846            ifds.append(page)
3847            index += 1
3848        shape = (len(ifds),) + pshape
3849        series.append(
3850            TiffPageSeries(
3851                ifds, shape, dtype, axes, name='Baseline', kind='QPI'
3852            )
3853        )
3854
3855        if index < len(pages):
3856            # Thumbnail
3857            page = pages[index]
3858            series.append(
3859                TiffPageSeries(
3860                    [page],
3861                    page.shape,
3862                    page.dtype,
3863                    page.axes,
3864                    name='Thumbnail',
3865                    kind='QPI',
3866                )
3867            )
3868            index += 1
3869
3870        if pages[0].is_tiled:
3871            # Resolutions
3872            while index < len(pages):
3873                pshape = (pshape[0] // 2, pshape[1] // 2) + pshape[2:]
3874                ifds = []
3875                while index < len(pages):
3876                    page = pages[index]
3877                    if page.shape != pshape:
3878                        break
3879                    ifds.append(page)
3880                    index += 1
3881                if len(ifds) != len(series[0].pages):
3882                    break
3883                shape = (len(ifds),) + pshape
3884                series[0].levels.append(
3885                    TiffPageSeries(
3886                        ifds, shape, dtype, axes, name='Resolution', kind='QPI'
3887                    )
3888                )
3889
3890        if series[0].is_pyramidal and index < len(pages):
3891            # Macro
3892            page = pages[index]
3893            series.append(
3894                TiffPageSeries(
3895                    [page],
3896                    page.shape,
3897                    page.dtype,
3898                    page.axes,
3899                    name='Macro',
3900                    kind='QPI',
3901                )
3902            )
3903            index += 1
3904            # Label
3905            if index < len(pages):
3906                page = pages[index]
3907                series.append(
3908                    TiffPageSeries(
3909                        [page],
3910                        page.shape,
3911                        page.dtype,
3912                        page.axes,
3913                        name='Label',
3914                        kind='QPI',
3915                    )
3916                )
3917
3918        self.is_uniform = False
3919        return series
3920
3921    def _series_svs(self):
3922        """Return image series in Aperio SVS file."""
3923        if not self.pages[0].is_tiled:
3924            return None
3925
3926        series = []
3927        self.is_uniform = False
3928        self.pages.cache = True
3929        self.pages.useframes = False
3930        self.pages.keyframe = 0
3931        self.pages._load()
3932
3933        # Baseline
3934        index = 0
3935        page = self.pages[index]
3936        series.append(
3937            TiffPageSeries(
3938                [page],
3939                page.shape,
3940                page.dtype,
3941                page.axes,
3942                name='Baseline',
3943                kind='SVS',
3944            )
3945        )
3946        # Thumbnail
3947        index += 1
3948        if index == len(self.pages):
3949            return series
3950        page = self.pages[index]
3951        series.append(
3952            TiffPageSeries(
3953                [page],
3954                page.shape,
3955                page.dtype,
3956                page.axes,
3957                name='Thumbnail',
3958                kind='SVS',
3959            )
3960        )
3961        # Resolutions
3962        # TODO: resolutions not by two
3963        index += 1
3964        while index < len(self.pages):
3965            page = self.pages[index]
3966            if not page.is_tiled or page.is_reduced:
3967                break
3968            series[0].levels.append(
3969                TiffPageSeries(
3970                    [page],
3971                    page.shape,
3972                    page.dtype,
3973                    page.axes,
3974                    name='Resolution',
3975                    kind='SVS',
3976                )
3977            )
3978            index += 1
3979        # Label, Macro; subfiletype 1, 9
3980        for name in ('Label', 'Macro'):
3981            if index == len(self.pages):
3982                break
3983            page = self.pages[index]
3984            series.append(
3985                TiffPageSeries(
3986                    [page],
3987                    page.shape,
3988                    page.dtype,
3989                    page.axes,
3990                    name=name,
3991                    kind='SVS',
3992                )
3993            )
3994            index += 1
3995
3996        return series
3997
3998    def _series_scn(self):
3999        """Return pyramidal image series in Leica SCN file."""
4000        # TODO: support collections
4001        from xml.etree import ElementTree as etree  # delayed import
4002
4003        scnxml = self.pages[0].description
4004        root = etree.fromstring(scnxml)
4005
4006        series = []
4007        self.is_uniform = False
4008        self.pages.cache = True
4009        self.pages.useframes = False
4010        self.pages.keyframe = 0
4011        self.pages._load()
4012
4013        for collection in root:
4014            if not collection.tag.endswith('collection'):
4015                continue
4016            for image in collection:
4017                if not image.tag.endswith('image'):
4018                    continue
4019                name = image.attrib.get('name', 'Unknown')
4020                for pixels in image:
4021                    if not pixels.tag.endswith('pixels'):
4022                        continue
4023                    resolutions = {}
4024                    for dimension in pixels:
4025                        if not dimension.tag.endswith('dimension'):
4026                            continue
4027                        if int(image.attrib.get('sizeZ', 1)) > 1:
4028                            raise NotImplementedError(
4029                                'SCN series: Z-Stacks not supported'
4030                            )
4031                        sizex = int(dimension.attrib['sizeX'])
4032                        sizey = int(dimension.attrib['sizeY'])
4033                        c = int(dimension.attrib.get('c', 0))
4034                        z = int(dimension.attrib.get('z', 0))
4035                        r = int(dimension.attrib.get('r', 0))
4036                        ifd = int(dimension.attrib['ifd'])
4037                        if r in resolutions:
4038                            level = resolutions[r]
4039                            if c > level['channels']:
4040                                level['channels'] = c
4041                            if z > level['sizez']:
4042                                level['sizez'] = z
4043                            level['ifds'][(c, z)] = ifd
4044                        else:
4045                            resolutions[r] = {
4046                                'size': [sizey, sizex],
4047                                'channels': c,
4048                                'sizez': z,
4049                                'ifds': {(c, z): ifd},
4050                            }
4051                    if not resolutions:
4052                        continue
4053                    levels = []
4054                    for r, level in sorted(resolutions.items()):
4055                        shape = (level['channels'] + 1, level['sizez'] + 1)
4056                        axes = 'CZ'
4057
4058                        ifds = [None] * product(shape)
4059                        for (c, z), ifd in sorted(level['ifds'].items()):
4060                            ifds[c * shape[1] + z] = self.pages[ifd]
4061
4062                        axes += ifds[0].axes
4063                        shape += ifds[0].shape
4064                        dtype = ifds[0].dtype
4065
4066                        levels.append(
4067                            TiffPageSeries(
4068                                ifds,
4069                                shape,
4070                                dtype,
4071                                axes,
4072                                parent=self,
4073                                name=name,
4074                                kind='SCN',
4075                            )
4076                        )
4077                    levels[0].levels.extend(levels[1:])
4078                    series.append(levels[0])
4079
4080        return series
4081
4082    def _series_bif(self):
4083        """Return image series in Ventana BIF file."""
4084        if not self.pages[0].is_tiled:
4085            return None
4086
4087        series = []
4088        baseline = None
4089        self.is_uniform = False
4090        self.pages.cache = True
4091        self.pages.useframes = False
4092        self.pages.keyframe = 0
4093        self.pages._load()
4094
4095        for page in self.pages:
4096            if page.description == 'Label Image':
4097                series.append(
4098                    TiffPageSeries(
4099                        [page],
4100                        page.shape,
4101                        page.dtype,
4102                        page.axes,
4103                        name='Label',
4104                        kind='BIF',
4105                    )
4106                )
4107            elif page.description == 'Thumbnail':
4108                series.append(
4109                    TiffPageSeries(
4110                        [page],
4111                        page.shape,
4112                        page.dtype,
4113                        page.axes,
4114                        name='Thumbnail',
4115                        kind='BIF',
4116                    )
4117                )
4118            elif 'level' not in page.description:
4119                # TODO: is this necessary?
4120                series.append(
4121                    TiffPageSeries(
4122                        [page],
4123                        page.shape,
4124                        page.dtype,
4125                        page.axes,
4126                        name='Unknown',
4127                        kind='BIF',
4128                    )
4129                )
4130            elif baseline is None:
4131                baseline = TiffPageSeries(
4132                    [page],
4133                    page.shape,
4134                    page.dtype,
4135                    page.axes,
4136                    name='Baseline',
4137                    kind='BIF',
4138                )
4139                series.insert(0, baseline)
4140            else:
4141                baseline.levels.append(
4142                    TiffPageSeries(
4143                        [page],
4144                        page.shape,
4145                        page.dtype,
4146                        page.axes,
4147                        name='Resolution',
4148                        kind='SVS',
4149                    )
4150                )
4151
4152        log_warning('BIF series: beware, tiles are not stiched')
4153        return series
4154
4155    def _series_ome(self):
4156        """Return image series in OME-TIFF file(s)."""
4157        # xml.etree found to be faster than lxml
4158        from xml.etree import ElementTree as etree  # delayed import
4159
4160        omexml = self.pages[0].description
4161        try:
4162            root = etree.fromstring(omexml)
4163        except etree.ParseError as exc:
4164            # TODO: test badly encoded OME-XML
4165            log_warning(f'OME series: {exc.__class__.__name__}: {exc}')
4166            try:
4167                omexml = omexml.decode(errors='ignore').encode()
4168                root = etree.fromstring(omexml)
4169            except Exception:
4170                return None
4171
4172        self.pages.cache = True
4173        self.pages.useframes = True
4174        self.pages.keyframe = 0
4175        self.pages._load(keyframe=None)
4176
4177        root_uuid = root.attrib.get('UUID', None)
4178        self._files = {root_uuid: self}
4179        dirname = self._fh.dirname
4180        moduloref = []
4181        modulo = {}
4182        series = []
4183        for element in root:
4184            if element.tag.endswith('BinaryOnly'):
4185                # TODO: load OME-XML from master or companion file
4186                log_warning('OME series: not an ome-tiff master file')
4187                break
4188            if element.tag.endswith('StructuredAnnotations'):
4189                for annot in element:
4190                    if not annot.attrib.get('Namespace', '').endswith(
4191                        'modulo'
4192                    ):
4193                        continue
4194                    modulo[annot.attrib['ID']] = mod = {}
4195                    for value in annot:
4196                        for modul in value:
4197                            for along in modul:
4198                                if not along.tag[:-1].endswith('Along'):
4199                                    continue
4200                                axis = along.tag[-1]
4201                                newaxis = along.attrib.get('Type', 'other')
4202                                newaxis = TIFF.AXES_LABELS[newaxis]
4203                                if 'Start' in along.attrib:
4204                                    step = float(along.attrib.get('Step', 1))
4205                                    start = float(along.attrib['Start'])
4206                                    stop = float(along.attrib['End']) + step
4207                                    labels = numpy.arange(start, stop, step)
4208                                else:
4209                                    labels = [
4210                                        label.text
4211                                        for label in along
4212                                        if label.tag.endswith('Label')
4213                                    ]
4214                                mod[axis] = (newaxis, labels)
4215
4216            if not element.tag.endswith('Image'):
4217                continue
4218
4219            for annot in element:
4220                if annot.tag.endswith('AnnotationRef'):
4221                    annotationref = annot.attrib['ID']
4222                    break
4223            else:
4224                annotationref = None
4225
4226            attr = element.attrib
4227            name = attr.get('Name', None)
4228
4229            for pixels in element:
4230                if not pixels.tag.endswith('Pixels'):
4231                    continue
4232                attr = pixels.attrib
4233                # dtype = attr.get('PixelType', None)
4234                axes = ''.join(reversed(attr['DimensionOrder']))
4235                shape = [int(attr['Size' + ax]) for ax in axes]
4236                ifds = []
4237                spp = 1  # samples per pixel
4238                first = True
4239
4240                for data in pixels:
4241                    if data.tag.endswith('Channel'):
4242                        attr = data.attrib
4243                        if first:
4244                            first = False
4245                            spp = int(attr.get('SamplesPerPixel', spp))
4246                            if spp > 1:
4247                                # correct channel dimension for spp
4248                                shape = [
4249                                    shape[i] // spp if ax == 'C' else shape[i]
4250                                    for i, ax in enumerate(axes)
4251                                ]
4252                        elif int(attr.get('SamplesPerPixel', 1)) != spp:
4253                            raise ValueError(
4254                                'OME series: cannot handle differing '
4255                                'SamplesPerPixel'
4256                            )
4257                        continue
4258
4259                    if not data.tag.endswith('TiffData'):
4260                        continue
4261
4262                    attr = data.attrib
4263                    ifd = int(attr.get('IFD', 0))
4264                    num = int(attr.get('NumPlanes', 1 if 'IFD' in attr else 0))
4265                    num = int(attr.get('PlaneCount', num))
4266                    idx = [int(attr.get('First' + ax, 0)) for ax in axes[:-2]]
4267                    try:
4268                        idx = int(numpy.ravel_multi_index(idx, shape[:-2]))
4269                    except ValueError:
4270                        # ImageJ produces invalid ome-xml when cropping
4271                        log_warning('OME series: invalid TiffData index')
4272                        continue
4273                    for uuid in data:
4274                        if not uuid.tag.endswith('UUID'):
4275                            continue
4276                        if root_uuid is None and uuid.text is not None:
4277                            # no global UUID, use this file
4278                            root_uuid = uuid.text
4279                            self._files[root_uuid] = self._files[None]
4280                        elif uuid.text not in self._files:
4281                            if not self._multifile:
4282                                # abort reading multifile OME series
4283                                # and fall back to generic series
4284                                return []
4285                            fname = uuid.attrib['FileName']
4286                            try:
4287                                tif = TiffFile(
4288                                    os.path.join(dirname, fname), _parent=self
4289                                )
4290                                tif.pages.cache = True
4291                                tif.pages.useframes = True
4292                                tif.pages.keyframe = 0
4293                                tif.pages._load(keyframe=None)
4294                            except (OSError, FileNotFoundError, ValueError):
4295                                log_warning(
4296                                    f'OME series: failed to read {fname!r}'
4297                                )
4298                                # assume that size is same as in previous file
4299                                # if no NumPlanes or PlaneCount are given
4300                                size = num if num else size  # noqa: undefined
4301                                ifds.extend([None] * (size + idx - len(ifds)))
4302                                break
4303                            self._files[uuid.text] = tif
4304                            tif.close()
4305                        pages = self._files[uuid.text].pages
4306                        try:
4307                            size = num if num else len(pages)
4308                            ifds.extend([None] * (size + idx - len(ifds)))
4309                            for i in range(size):
4310                                ifds[idx + i] = pages[ifd + i]
4311                        except IndexError:
4312                            log_warning('OME series: index out of range')
4313                        # only process first UUID
4314                        break
4315                    else:
4316                        # no uuid found
4317                        pages = self.pages
4318                        try:
4319                            size = num if num else len(pages)
4320                            ifds.extend([None] * (size + idx - len(ifds)))
4321                            for i in range(size):
4322                                ifds[idx + i] = pages[ifd + i]
4323                        except IndexError:
4324                            log_warning('OME series: index out of range')
4325
4326                if not ifds or all(i is None for i in ifds):
4327                    # skip images without data
4328                    continue
4329
4330                # find a keyframe
4331                keyframe = None
4332                for ifd in ifds:
4333                    # try find a TiffPage
4334                    if ifd and ifd == ifd.keyframe:
4335                        keyframe = ifd
4336                        break
4337                if keyframe is None:
4338                    # reload a TiffPage from file
4339                    for i, keyframe in enumerate(ifds):
4340                        if keyframe:
4341                            isclosed = keyframe.parent.filehandle.closed
4342                            if isclosed:
4343                                keyframe.parent.filehandle.open()
4344                            keyframe.parent.pages.keyframe = keyframe.index
4345                            keyframe = keyframe.parent.pages[keyframe.index]
4346                            ifds[i] = keyframe
4347                            if isclosed:
4348                                keyframe.parent.filehandle.close()
4349                            break
4350
4351                # does the series spawn multiple files
4352                multifile = False
4353                for ifd in ifds:
4354                    if ifd and ifd.parent != keyframe.parent:
4355                        multifile = True
4356                        break
4357
4358                if spp > 1:
4359                    if keyframe.planarconfig == 1:
4360                        shape += [spp]
4361                        axes += 'S'
4362                    else:
4363                        shape = shape[:-2] + [spp] + shape[-2:]
4364                        axes = axes[:-2] + 'S' + axes[-2:]
4365                if 'S' not in shape:
4366                    shape += [1]
4367                    axes += 'S'
4368
4369                # there might be more pages in the file than referenced in XML
4370                # e.g. Nikon-cell011.ome.tif
4371                size = max(product(shape) // keyframe.size, 1)
4372                if size != len(ifds):
4373                    log_warning(
4374                        'OME series: expected %s frames, got %s',
4375                        size,
4376                        len(ifds),
4377                    )
4378                    ifds = ifds[:size]
4379
4380                # FIXME: this implementation assumes the last dimensions are
4381                # stored in TIFF pages. Apparently that is not always the case.
4382                # E.g. TCX (20000, 2, 500) is stored in 2 pages of (20000, 500)
4383                # in 'Image 7.ome_h00.tiff'.
4384                # For now, verify that shapes of keyframe and series match.
4385                # If not, skip series.
4386                squeezed = squeeze_axes(shape, axes)[0]
4387                if keyframe.shape != tuple(squeezed[-len(keyframe.shape) :]):
4388                    log_warning(
4389                        'OME series: cannot handle discontiguous storage '
4390                        '%s != %s',
4391                        keyframe.shape,
4392                        tuple(squeezed[-len(keyframe.shape) :]),
4393                    )
4394                    del ifds
4395                    continue
4396
4397                # set keyframe on all IFDs
4398                keyframes = {keyframe.parent.filehandle.name: keyframe}
4399                for i, page in enumerate(ifds):
4400                    if page is None:
4401                        continue
4402                    fh = page.parent.filehandle
4403                    if fh.name not in keyframes:
4404                        if page.keyframe != page:
4405                            # reload TiffPage from file
4406                            isclosed = fh.closed
4407                            if isclosed:
4408                                fh.open()
4409                            page.parent.pages.keyframe = page.index
4410                            page = page.parent.pages[page.index]
4411                            ifds[i] = page
4412                            if isclosed:
4413                                fh.close()
4414                        keyframes[fh.name] = page
4415                    if page.keyframe != page:
4416                        page.keyframe = keyframes[fh.name]
4417
4418                moduloref.append(annotationref)
4419                series.append(
4420                    TiffPageSeries(
4421                        ifds,
4422                        shape,
4423                        keyframe.dtype,
4424                        axes,
4425                        parent=self,
4426                        name=name,
4427                        multifile=multifile,
4428                        kind='OME',
4429                    )
4430                )
4431                del ifds
4432
4433        for serie, annotationref in zip(series, moduloref):
4434            if annotationref not in modulo:
4435                continue
4436            shape = list(serie.get_shape(False))
4437            axes = serie.get_axes(False)
4438            for axis, (newaxis, labels) in modulo[annotationref].items():
4439                i = axes.index(axis)
4440                size = len(labels)
4441                if shape[i] == size:
4442                    axes = axes.replace(axis, newaxis, 1)
4443                else:
4444                    shape[i] //= size
4445                    shape.insert(i + 1, size)
4446                    axes = axes.replace(axis, axis + newaxis, 1)
4447            serie.set_shape_axes(shape, axes)
4448
4449        # pyramids
4450        for serie in series:
4451            keyframe = serie.keyframe
4452            if keyframe.subifds is None:
4453                continue
4454            if len(self._files) > 1:
4455                # TODO: support multi-file pyramids; must re-open/close
4456                log_warning('OME series: cannot read multi-file pyramids')
4457                break
4458            for level in range(len(keyframe.subifds)):
4459                keyframe = None
4460                ifds = []
4461                for page in serie.pages:
4462                    if page is None:
4463                        ifds.append(None)
4464                        continue
4465                    page.parent.filehandle.seek(page.subifds[level])
4466                    if page.keyframe == page:
4467                        ifd = keyframe = TiffPage(self, (page.index, level))
4468                    elif keyframe is None:
4469                        raise RuntimeError('no keyframe')
4470                    else:
4471                        ifd = TiffFrame(self, page.index, keyframe=keyframe)
4472                    ifds.append(ifd)
4473                # fix shape
4474                shape = []
4475                for i, ax in enumerate(serie.axes):
4476                    if ax == 'X':
4477                        shape.append(keyframe.imagewidth)
4478                    elif ax == 'Y':
4479                        shape.append(keyframe.imagelength)
4480                    else:
4481                        shape.append(serie.shape[i])
4482                # add series
4483                serie.levels.append(
4484                    TiffPageSeries(
4485                        ifds,
4486                        tuple(shape),
4487                        keyframe.dtype,
4488                        serie.axes,
4489                        parent=self,
4490                        name=f'level {level + 1}',
4491                        kind='OME',
4492                    )
4493                )
4494
4495        self.is_uniform = len(series) == 1 and len(series[0].levels) == 1
4496
4497        return series
4498
4499    def _series_stk(self):
4500        """Return series in STK file."""
4501        page = self.pages[0]
4502        meta = self.stk_metadata
4503        planes = meta['NumberPlanes']
4504        name = meta.get('Name', '')
4505        if planes == 1:
4506            shape = (1,) + page.shape
4507            axes = 'I' + page.axes
4508        elif numpy.all(meta['ZDistance'] != 0):
4509            shape = (planes,) + page.shape
4510            axes = 'Z' + page.axes
4511        elif numpy.all(numpy.diff(meta['TimeCreated']) != 0):
4512            shape = (planes,) + page.shape
4513            axes = 'T' + page.axes
4514        else:
4515            # TODO: determine other/combinations of dimensions
4516            shape = (planes,) + page.shape
4517            axes = 'I' + page.axes
4518        self.is_uniform = True
4519        series = TiffPageSeries(
4520            [page],
4521            shape,
4522            page.dtype,
4523            axes,
4524            name=name,
4525            truncated=planes > 1,
4526            kind='STK',
4527        )
4528        return [series]
4529
4530    def _series_lsm(self):
4531        """Return main and thumbnail series in LSM file."""
4532        lsmi = self.lsm_metadata
4533        axes = TIFF.CZ_LSMINFO_SCANTYPE[lsmi['ScanType']]
4534        if self.pages[0].photometric == 2:  # RGB; more than one channel
4535            axes = axes.replace('C', '').replace('XY', 'XYC')
4536        if lsmi.get('DimensionP', 0) > 0:
4537            axes += 'P'
4538        if lsmi.get('DimensionM', 0) > 0:
4539            axes += 'M'
4540        axes = axes[::-1]
4541        shape = tuple(int(lsmi[TIFF.CZ_LSMINFO_DIMENSIONS[i]]) for i in axes)
4542
4543        name = lsmi.get('Name', '')
4544        pages = self.pages._getlist(slice(0, None, 2), validate=False)
4545        dtype = pages[0].dtype
4546        series = [
4547            TiffPageSeries(pages, shape, dtype, axes, name=name, kind='LSM')
4548        ]
4549
4550        page = self.pages[1]
4551        if page.is_reduced:
4552            pages = self.pages._getlist(slice(1, None, 2), validate=False)
4553            dtype = page.dtype
4554            cp = 1
4555            i = 0
4556            while cp < len(pages) and i < len(shape) - 2:
4557                cp *= shape[i]
4558                i += 1
4559            shape = shape[:i] + page.shape
4560            axes = axes[:i] + 'SYX'
4561            series.append(
4562                TiffPageSeries(
4563                    pages, shape, dtype, axes, name=name, kind='LSMreduced'
4564                )
4565            )
4566
4567        self.is_uniform = False
4568        return series
4569
4570    def _lsm_load_pages(self):
4571        """Load and fix all pages from LSM file."""
4572        # cache all pages to preserve corrected values
4573        pages = self.pages
4574        pages.cache = True
4575        pages.useframes = True
4576        # use first and second page as keyframes
4577        pages.keyframe = 1
4578        pages.keyframe = 0
4579        # load remaining pages as frames
4580        pages._load(keyframe=None)
4581        # fix offsets and bytecounts first
4582        # TODO: fix multiple conversions between lists and tuples
4583        self._lsm_fix_strip_offsets()
4584        self._lsm_fix_strip_bytecounts()
4585        # assign keyframes for data and thumbnail series
4586        keyframe = pages[0]
4587        for page in pages[::2]:
4588            page.keyframe = keyframe
4589        keyframe = pages[1]
4590        for page in pages[1::2]:
4591            page.keyframe = keyframe
4592
4593    def _lsm_fix_strip_offsets(self):
4594        """Unwrap strip offsets for LSM files greater than 4 GB.
4595
4596        Each series and position require separate unwrapping (undocumented).
4597
4598        """
4599        if self.filehandle.size < 2 ** 32:
4600            return
4601
4602        pages = self.pages
4603        npages = len(pages)
4604        series = self.series[0]
4605        axes = series.axes
4606
4607        # find positions
4608        positions = 1
4609        for i in 0, 1:
4610            if series.axes[i] in 'PM':
4611                positions *= series.shape[i]
4612
4613        # make time axis first
4614        if positions > 1:
4615            ntimes = 0
4616            for i in 1, 2:
4617                if axes[i] == 'T':
4618                    ntimes = series.shape[i]
4619                    break
4620            if ntimes:
4621                div, mod = divmod(npages, 2 * positions * ntimes)
4622                if mod != 0:
4623                    raise RuntimeError('mod != 0')
4624                shape = (positions, ntimes, div, 2)
4625                indices = numpy.arange(product(shape)).reshape(shape)
4626                indices = numpy.moveaxis(indices, 1, 0)
4627        else:
4628            indices = numpy.arange(npages).reshape(-1, 2)
4629
4630        # images of reduced page might be stored first
4631        if pages[0].dataoffsets[0] > pages[1].dataoffsets[0]:
4632            indices = indices[..., ::-1]
4633
4634        # unwrap offsets
4635        wrap = 0
4636        previousoffset = 0
4637        for i in indices.flat:
4638            page = pages[int(i)]
4639            dataoffsets = []
4640            for currentoffset in page.dataoffsets:
4641                if currentoffset < previousoffset:
4642                    wrap += 2 ** 32
4643                dataoffsets.append(currentoffset + wrap)
4644                previousoffset = currentoffset
4645            page.dataoffsets = tuple(dataoffsets)
4646
4647    def _lsm_fix_strip_bytecounts(self):
4648        """Set databytecounts to size of compressed data.
4649
4650        The StripByteCounts tag in LSM files contains the number of bytes
4651        for the uncompressed data.
4652
4653        """
4654        pages = self.pages
4655        if pages[0].compression == 1:
4656            return
4657        # sort pages by first strip offset
4658        pages = sorted(pages, key=lambda p: p.dataoffsets[0])
4659        npages = len(pages) - 1
4660        for i, page in enumerate(pages):
4661            if page.index % 2:
4662                continue
4663            offsets = page.dataoffsets
4664            bytecounts = page.databytecounts
4665            if i < npages:
4666                lastoffset = pages[i + 1].dataoffsets[0]
4667            else:
4668                # LZW compressed strips might be longer than uncompressed
4669                lastoffset = min(
4670                    offsets[-1] + 2 * bytecounts[-1], self._fh.size
4671                )
4672            bytecounts = list(bytecounts)
4673            for j in range(len(bytecounts) - 1):
4674                bytecounts[j] = offsets[j + 1] - offsets[j]
4675            bytecounts[-1] = lastoffset - offsets[-1]
4676            page.databytecounts = tuple(bytecounts)
4677
4678    def _ndpi_load_pages(self):
4679        """Load and fix pages from NDPI slide file if CaptureMode > 6.
4680
4681        If the value of the CaptureMode tag is greater than 6, change the
4682        attributes of the TiffPages that are part of the pyramid to match
4683        16-bit grayscale data. TiffTags are not corrected.
4684
4685        """
4686        pages = self.pages
4687        capturemode = pages[0].tags.get(65441, None)
4688        if capturemode is None or capturemode.value < 6:
4689            return
4690
4691        pages.cache = True
4692        pages.useframes = False
4693        pages._load()
4694
4695        for page in pages:
4696            mag = page.tags.get(65421, None)
4697            if mag is None or mag.value > 0:
4698                page.photometric = TIFF.PHOTOMETRIC.MINISBLACK
4699                page.samplesperpixel = 1
4700                page.sampleformat = 1
4701                page.bitspersample = 16
4702                page.dtype = page._dtype = numpy.dtype('uint16')
4703                if page.shaped[-1] > 1:
4704                    page.axes = page.axes[:-1]
4705                    page.shape = page.shape[:-1]
4706                    page.shaped = page.shaped[:-1] + (1,)
4707
4708    def _philips_load_pages(self):
4709        """Load and fix all pages from Philips slide file.
4710
4711        The imagewidth and imagelength values of all tiled pages are corrected
4712        using the DICOM_PIXEL_SPACING attributes of the XML formatted
4713        description of the first page.
4714
4715        """
4716        from xml.etree import ElementTree as etree  # delayed import
4717
4718        pages = self.pages
4719        pages.cache = True
4720        pages.useframes = False
4721        pages._load()
4722        npages = len(pages)
4723
4724        root = etree.fromstring(pages[0].description)
4725
4726        imagewidth = pages[0].imagewidth
4727        imagelength = pages[0].imagelength
4728        sizes = None
4729        for elem in root.iter():
4730            if (
4731                elem.tag != 'Attribute'
4732                or elem.attrib['Name'] != 'DICOM_PIXEL_SPACING'
4733            ):
4734                continue
4735            w, h = (float(v) for v in elem.text.replace('"', '').split())
4736            if sizes is None:
4737                imagelength *= h
4738                imagewidth *= w
4739                sizes = []
4740            else:
4741                sizes.append(
4742                    (
4743                        int(math.ceil(imagelength / h)),
4744                        int(math.ceil(imagewidth / w)),
4745                    )
4746                )
4747
4748        i = 0
4749        for imagelength, imagewidth in sizes:
4750            while i < npages and pages[i].tilewidth == 0:
4751                # Label, Macro
4752                i += 1
4753                continue
4754            if i == npages:
4755                break
4756            page = pages[i]
4757            page.imagewidth = imagewidth
4758            page.imagelength = imagelength
4759            if page.shaped[-1] > 1:
4760                page.shape = (imagelength, imagewidth, page.shape[-1])
4761            elif page.shaped[0] > 1:
4762                page.shape = (page.shape[0], imagelength, imagewidth)
4763            else:
4764                page.shape = (imagelength, imagewidth)
4765            page.shaped = (
4766                page.shaped[:2] + (imagelength, imagewidth) + page.shaped[-1:]
4767            )
4768            i += 1
4769
4770    def __getattr__(self, name):
4771        """Return 'is_flag' attributes from first page."""
4772        if name[3:] in TIFF.FILE_FLAGS:
4773            if not self.pages:
4774                return False
4775            value = bool(getattr(self.pages[0], name))
4776            setattr(self, name, value)
4777            return value
4778        raise AttributeError(
4779            f'{self.__class__.__name__!r} object has no attribute {name!r}'
4780        )
4781
4782    def __enter__(self):
4783        return self
4784
4785    def __exit__(self, exc_type, exc_value, traceback):
4786        self.close()
4787
4788    def __str__(self, detail=0, width=79):
4789        """Return string containing information about TiffFile.
4790
4791        The detail parameter specifies the level of detail returned:
4792
4793        0: file only.
4794        1: all series, first page of series and its tags.
4795        2: large tag values and file metadata.
4796        3: all pages.
4797
4798        """
4799        info = [
4800            "TiffFile '{}'",
4801            format_size(self._fh.size),
4802            ''
4803            if byteorder_isnative(self.byteorder)
4804            else {'<': 'little-endian', '>': 'big-endian'}[self.byteorder],
4805        ]
4806        if self.is_bigtiff:
4807            info.append('BigTiff')
4808        info.append(' '.join(f.lower() for f in self.flags))
4809        if len(self.pages) > 1:
4810            info.append(f'{len(self.pages)} Pages')
4811        if len(self.series) > 1:
4812            info.append(f'{len(self.series)} Series')
4813        if len(self._files) > 1:
4814            info.append(f'{len(self._files)} Files')
4815        info = '  '.join(info)
4816        info = info.replace('    ', '  ').replace('   ', '  ')
4817        info = info.format(
4818            snipstr(self._fh.name, max(12, width + 2 - len(info)))
4819        )
4820        if detail <= 0:
4821            return info
4822        info = [info]
4823        info.append('\n'.join(str(s) for s in self.series))
4824        if detail >= 3:
4825            for p in self.pages:
4826                if p is None:
4827                    continue
4828                info.append(TiffPage.__str__(p, detail=detail, width=width))
4829                for s in p.pages:
4830                    info.append(
4831                        TiffPage.__str__(s, detail=detail, width=width)
4832                    )
4833        elif self.series:
4834            info.extend(
4835                TiffPage.__str__(s.pages[0], detail=detail, width=width)
4836                for s in self.series
4837                if s.pages[0] is not None
4838            )
4839        elif self.pages and self.pages[0]:
4840            info.append(
4841                TiffPage.__str__(self.pages[0], detail=detail, width=width)
4842            )
4843        if detail >= 2:
4844            for name in sorted(self.flags):
4845                if hasattr(self, name + '_metadata'):
4846                    m = getattr(self, name + '_metadata')
4847                    if m:
4848                        info.append(
4849                            '{}_METADATA\n{}'.format(
4850                                name.upper(),
4851                                pformat(m, width=width, height=detail * 24),
4852                            )
4853                        )
4854        return '\n\n'.join(info).replace('\n\n\n', '\n\n')
4855
4856    @lazyattr
4857    def flags(self):
4858        """Return set of file flags, a potentially expensive operation."""
4859        return {
4860            name.lower()
4861            for name in sorted(TIFF.FILE_FLAGS)
4862            if getattr(self, 'is_' + name)
4863        }
4864
4865    @property
4866    def is_bigtiff(self):
4867        """Return if file has BigTIFF format."""
4868        return self.tiff.version == 43
4869
4870    @lazyattr
4871    def is_mdgel(self):
4872        """Return if file has MD Gel format."""
4873        # side effect: add second page, if exists, to cache
4874        try:
4875            ismdgel = (
4876                self.pages[0].is_mdgel
4877                or self.pages.get(1, cache=True).is_mdgel
4878            )
4879            if ismdgel:
4880                self.is_uniform = False
4881            return ismdgel
4882        except IndexError:
4883            return False
4884
4885    @lazyattr
4886    def is_uniform(self):
4887        """Return if file contains a uniform series of pages."""
4888        # the hashes of IFDs 0, 7, and -1 are the same
4889        pages = self.pages
4890        page = pages[0]
4891        if page.subifds:
4892            return False
4893        if page.is_scanimage or page.is_nih:
4894            return True
4895        try:
4896            useframes = pages.useframes
4897            pages.useframes = False
4898            h = page.hash
4899            for i in (1, 7, -1):
4900                if pages[i].aspage().hash != h:
4901                    return False
4902        except IndexError:
4903            return False
4904        finally:
4905            pages.useframes = useframes
4906        return True
4907
4908    @property
4909    def is_appendable(self):
4910        """Return if pages can be appended to file without corrupting."""
4911        # TODO: check other formats
4912        return not (
4913            self.is_ome
4914            or self.is_lsm
4915            or self.is_stk
4916            or self.is_imagej
4917            or self.is_fluoview
4918            or self.is_micromanager
4919        )
4920
4921    @lazyattr
4922    def shaped_metadata(self):
4923        """Return tifffile metadata from JSON descriptions as dicts."""
4924        if not self.is_shaped:
4925            return None
4926        return tuple(
4927            json_description_metadata(s.pages[0].is_shaped)
4928            for s in self.series
4929            if s.kind.lower() == 'shaped'
4930        )
4931
4932    @property
4933    def ome_metadata(self):
4934        """Return OME XML."""
4935        if not self.is_ome:
4936            return None
4937        # return xml2dict(self.pages[0].description)['OME']
4938        return self.pages[0].description
4939
4940    @property
4941    def scn_metadata(self):
4942        """Return Leica SCN XML."""
4943        if not self.is_scn:
4944            return None
4945        return self.pages[0].description
4946
4947    @property
4948    def philips_metadata(self):
4949        """Return Philips DP XML."""
4950        if not self.is_philips:
4951            return None
4952        return self.pages[0].description
4953
4954    @property
4955    def lsm_metadata(self):
4956        """Return LSM metadata from CZ_LSMINFO tag as dict."""
4957        if not self.is_lsm:
4958            return None
4959        return self.pages[0].tags[34412].value  # CZ_LSMINFO
4960
4961    @lazyattr
4962    def stk_metadata(self):
4963        """Return STK metadata from UIC tags as dict."""
4964        if not self.is_stk:
4965            return None
4966        page = self.pages[0]
4967        result = {}
4968        result['NumberPlanes'] = page.tags[33629].count  # UIC2tag
4969        if page.description:
4970            result['PlaneDescriptions'] = page.description.split('\x00')
4971            # result['plane_descriptions'] = stk_description_metadata(
4972            #    page.image_description)
4973        tag = page.tags.get(33628)  # UIC1tag
4974        if tag is not None:
4975            result.update(tag.value)
4976        tag = page.tags.get(33630)  # UIC3tag
4977        if tag is not None:
4978            result.update(tag.value)  # wavelengths
4979        tag = page.tags.get(33631)  # UIC4tag
4980        if tag is not None:
4981            result.update(tag.value)  # override UIC1 tags
4982        uic2tag = page.tags[33629].value
4983        result['ZDistance'] = uic2tag['ZDistance']
4984        result['TimeCreated'] = uic2tag['TimeCreated']
4985        result['TimeModified'] = uic2tag['TimeModified']
4986        try:
4987            result['DatetimeCreated'] = numpy.array(
4988                [
4989                    julian_datetime(*dt)
4990                    for dt in zip(
4991                        uic2tag['DateCreated'], uic2tag['TimeCreated']
4992                    )
4993                ],
4994                dtype='datetime64[ns]',
4995            )
4996            result['DatetimeModified'] = numpy.array(
4997                [
4998                    julian_datetime(*dt)
4999                    for dt in zip(
5000                        uic2tag['DateModified'], uic2tag['TimeModified']
5001                    )
5002                ],
5003                dtype='datetime64[ns]',
5004            )
5005        except ValueError as exc:
5006            log_warning(f'STK metadata: {exc.__class__.__name__}: {exc}')
5007        return result
5008
5009    @lazyattr
5010    def imagej_metadata(self):
5011        """Return consolidated ImageJ metadata as dict."""
5012        if not self.is_imagej:
5013            return None
5014        page = self.pages[0]
5015        result = imagej_description_metadata(page.is_imagej)
5016        tag = page.tags.get(50839)  # IJMetadata
5017        if tag is not None:
5018            try:
5019                result.update(tag.value)
5020            except Exception:
5021                pass
5022        return result
5023
5024    @lazyattr
5025    def fluoview_metadata(self):
5026        """Return consolidated FluoView metadata as dict."""
5027        if not self.is_fluoview:
5028            return None
5029        result = {}
5030        page = self.pages[0]
5031        result.update(page.tags[34361].value)  # MM_Header
5032        # TODO: read stamps from all pages
5033        result['Stamp'] = page.tags[34362].value  # MM_Stamp
5034        # skip parsing image description; not reliable
5035        # try:
5036        #     t = fluoview_description_metadata(page.image_description)
5037        #     if t is not None:
5038        #         result['ImageDescription'] = t
5039        # except Exception as exc:
5040        #     log_warning(
5041        #         'FluoView metadata: failed to parse image description '
5042        #         f'({exc})'
5043        #     )
5044        return result
5045
5046    @property
5047    def nih_metadata(self):
5048        """Return NIH Image metadata from NIHImageHeader tag as dict."""
5049        if not self.is_nih:
5050            return None
5051        return self.pages[0].tags[43314].value  # NIHImageHeader
5052
5053    @property
5054    def fei_metadata(self):
5055        """Return FEI metadata from SFEG or HELIOS tags as dict."""
5056        if not self.is_fei:
5057            return None
5058        tags = self.pages[0].tags
5059        tag = tags.get(34680, tags.get(34682))  # FEI_SFEG or FEI_HELIOS
5060        return None if tag is None else tag.value
5061
5062    @property
5063    def sem_metadata(self):
5064        """Return SEM metadata from CZ_SEM tag as dict."""
5065        if not self.is_sem:
5066            return None
5067        return self.pages[0].tags[34118].value
5068
5069    @property
5070    def sis_metadata(self):
5071        """Return Olympus SIS metadata from SIS and INI tags as dict."""
5072        if not self.is_sis:
5073            return None
5074        tags = self.pages[0].tags
5075        result = {}
5076        try:
5077            result.update(tags[33471].value)  # OlympusINI
5078        except Exception:
5079            pass
5080        try:
5081            result.update(tags[33560].value)  # OlympusSIS
5082        except Exception:
5083            pass
5084        return result
5085
5086    @lazyattr
5087    def mdgel_metadata(self):
5088        """Return consolidated metadata from MD GEL tags as dict."""
5089        for page in self.pages[:2]:
5090            if 33445 in page.tags:  # MDFileTag
5091                tags = page.tags
5092                break
5093        else:
5094            return None
5095        result = {}
5096        for code in range(33445, 33453):
5097            if code not in tags:
5098                continue
5099            name = TIFF.TAGS[code]
5100            result[name[2:]] = tags[code].value
5101        return result
5102
5103    @property
5104    def andor_metadata(self):
5105        """Return Andor tags as dict."""
5106        return self.pages[0].andor_tags
5107
5108    @property
5109    def epics_metadata(self):
5110        """Return EPICS areaDetector tags as dict."""
5111        return self.pages[0].epics_tags
5112
5113    @property
5114    def tvips_metadata(self):
5115        """Return TVIPS tag as dict."""
5116        if not self.is_tvips:
5117            return None
5118        return self.pages[0].tags[37706].value
5119
5120    @lazyattr
5121    def metaseries_metadata(self):
5122        """Return MetaSeries metadata from image description as dict."""
5123        if not self.is_metaseries:
5124            return None
5125        return metaseries_description_metadata(self.pages[0].description)
5126
5127    @lazyattr
5128    def pilatus_metadata(self):
5129        """Return Pilatus metadata from image description as dict."""
5130        if not self.is_pilatus:
5131            return None
5132        return pilatus_description_metadata(self.pages[0].description)
5133
5134    @lazyattr
5135    def micromanager_metadata(self):
5136        """Return MicroManager non-TIFF settings from file as dict."""
5137        if not self.is_micromanager:
5138            return None
5139        # from file header
5140        return read_micromanager_metadata(self._fh)
5141
5142    @lazyattr
5143    def scanimage_metadata(self):
5144        """Return ScanImage non-varying frame and ROI metadata as dict.
5145
5146        The returned dict may be empty or contain 'FrameData', 'RoiGroups',
5147        and 'version' keys.
5148
5149        The varying frame data can be found in the ImageDescription tags.
5150
5151        """
5152        if not self.is_scanimage:
5153            return None
5154        result = {}
5155        try:
5156            framedata, roidata, version = read_scanimage_metadata(self._fh)
5157            result['version'] = version
5158            result['FrameData'] = framedata
5159            result.update(roidata)
5160        except ValueError:
5161            pass
5162        return result
5163
5164    @property
5165    def geotiff_metadata(self):
5166        """Return GeoTIFF metadata from first page as dict."""
5167        if not self.is_geotiff:
5168            return None
5169        return self.pages[0].geotiff_tags
5170
5171    @property
5172    def eer_metadata(self):
5173        """Return EER metadata from first page as XML."""
5174        if not self.is_eer:
5175            return None
5176        return self.pages[0].tags[65001].value.decode()
5177
5178
5179class TiffPages:
5180    """Sequence of TIFF image file directories (IFD chain).
5181
5182    Instances of TiffPages have a state (cache, keyframe, etc.) and are not
5183    thread-safe.
5184
5185    """
5186
5187    def __init__(self, arg, index=None):
5188        """Initialize instance and read first TiffPage from file.
5189
5190        If arg is a TiffFile, the file position must be at an offset to an
5191        offset to a TiffPage. If arg is a TiffPage, page offsets are read
5192        from the SubIFDs tag.
5193
5194        """
5195        self.parent = None
5196        self.pages = []  # cache of TiffPages, TiffFrames, or their offsets
5197        self._indexed = False  # True if offsets to all pages were read
5198        self._cached = False  # True if all pages were read into cache
5199        self._tiffpage = TiffPage  # class used for reading pages
5200        self._keyframe = None  # page that is currently used as keyframe
5201        self._cache = False  # do not cache frames or pages (if not keyframe)
5202        self._nextpageoffset = None
5203        self._index = (index,) if isinstance(index, int) else index
5204
5205        if isinstance(arg, TiffFile):
5206            # read offset to first page from current file position
5207            self.parent = arg
5208            fh = self.parent.filehandle
5209            self._nextpageoffset = fh.tell()
5210            offset = struct.unpack(
5211                self.parent.tiff.offsetformat,
5212                fh.read(self.parent.tiff.offsetsize),
5213            )[0]
5214            if offset == 0:
5215                log_warning('TiffPages: file contains no pages')
5216                self._indexed = True
5217                return
5218        elif 330 in arg.tags:
5219            # use offsets from SubIFDs tag
5220            self.parent = arg.parent
5221            fh = self.parent.filehandle
5222            offsets = arg.tags[330].value
5223            offset = offsets[0]
5224            if offset == 0:
5225                log_warning('TiffPages: TiffPage contains invalid SubIFDs')
5226                self._indexed = True
5227                return
5228        else:
5229            self._indexed = True
5230            return
5231
5232        if offset >= fh.size:
5233            log_warning(f'TiffPages: invalid page offset {offset!r}')
5234            self._indexed = True
5235            return
5236
5237        pageindex = 0 if self._index is None else self._index + (0,)
5238
5239        # read and cache first page
5240        fh.seek(offset)
5241        page = TiffPage(self.parent, index=pageindex)
5242        self.pages.append(page)
5243        self._keyframe = page
5244        if self._nextpageoffset is None:
5245            # offsets from SubIFDs tag
5246            self.pages.extend(offsets[1:])
5247            self._indexed = True
5248            self._cached = True
5249
5250    @property
5251    def cache(self):
5252        """Return if pages/frames are currently being cached."""
5253        return self._cache
5254
5255    @cache.setter
5256    def cache(self, value):
5257        """Enable or disable caching of pages/frames. Clear cache if False."""
5258        value = bool(value)
5259        if self._cache and not value:
5260            self._clear()
5261        self._cache = value
5262
5263    @property
5264    def useframes(self):
5265        """Return if currently using TiffFrame (True) or TiffPage (False)."""
5266        return self._tiffpage == TiffFrame and TiffFrame is not TiffPage
5267
5268    @useframes.setter
5269    def useframes(self, value):
5270        """Set to use TiffFrame (True) or TiffPage (False)."""
5271        self._tiffpage = TiffFrame if value else TiffPage
5272
5273    @property
5274    def keyframe(self):
5275        """Return current keyframe."""
5276        return self._keyframe
5277
5278    @keyframe.setter
5279    def keyframe(self, index):
5280        """Set current keyframe. Load TiffPage from file if necessary."""
5281        index = int(index)
5282        if index < 0:
5283            index %= len(self)
5284        if self._keyframe.index == index:
5285            return
5286        if index == 0:
5287            self._keyframe = self.pages[0]
5288            return
5289        if self._indexed or index < len(self.pages):
5290            page = self.pages[index]
5291            if isinstance(page, TiffPage):
5292                self._keyframe = page
5293                return
5294            if isinstance(page, TiffFrame):
5295                # remove existing TiffFrame
5296                self.pages[index] = page.offset
5297        # load TiffPage from file
5298        tiffpage = self._tiffpage
5299        self._tiffpage = TiffPage
5300        try:
5301            self._keyframe = self._getitem(index)
5302        finally:
5303            self._tiffpage = tiffpage
5304        # always cache keyframes
5305        self.pages[index] = self._keyframe
5306
5307    @property
5308    def next_page_offset(self):
5309        """Return offset where offset to a new page can be stored."""
5310        if not self._indexed:
5311            self._seek(-1)
5312        return self._nextpageoffset
5313
5314    def get(self, key, default=None, validate=False, cache=None, aspage=True):
5315        """Return specified page from cache or file."""
5316        try:
5317            return self._getitem(
5318                key, validate=validate, cache=cache, aspage=aspage
5319            )
5320        except IndexError:
5321            if default is None:
5322                raise
5323        return default
5324
5325    def _load(self, keyframe=True):
5326        """Read all remaining pages from file."""
5327        if self._cached:
5328            return
5329        pages = self.pages
5330        if not pages:
5331            return
5332        if not self._indexed:
5333            self._seek(-1)
5334        if not self._cache:
5335            return
5336        fh = self.parent.filehandle
5337        if keyframe is not None:
5338            keyframe = self._keyframe
5339        for i, page in enumerate(pages):
5340            if isinstance(page, (int, numpy.integer)):
5341                pageindex = i if self._index is None else self._index + (i,)
5342                fh.seek(page)
5343                page = self._tiffpage(
5344                    self.parent, index=pageindex, keyframe=keyframe
5345                )
5346                pages[i] = page
5347        self._cached = True
5348
5349    def _load_virtual_frames(self):
5350        """Calculate virtual TiffFrames."""
5351        pages = self.pages
5352        try:
5353            if len(pages) > 1:
5354                raise ValueError('pages already loaded')
5355            page = pages[0]
5356            if not page.is_contiguous:
5357                raise ValueError('data not contiguous')
5358            self._seek(4)
5359            delta = pages[2] - pages[1]
5360            if pages[3] - pages[2] != delta or pages[4] - pages[3] != delta:
5361                raise ValueError('page offsets not equidistant')
5362            page1 = self._getitem(1, validate=page.hash)
5363            offsetoffset = page1.dataoffsets[0] - page1.offset
5364            if offsetoffset < 0 or offsetoffset > delta:
5365                raise ValueError('page offsets not equidistant')
5366            pages = [page, page1]
5367            filesize = self.parent.filehandle.size - delta
5368
5369            for index, offset in enumerate(
5370                range(page1.offset + delta, filesize, delta)
5371            ):
5372                pageindex = index + 2
5373                d = pageindex * delta
5374                offsets = tuple(i + d for i in page.dataoffsets)
5375                offset = offset if offset < 2 ** 31 - 1 else None
5376                if self._index is not None:
5377                    pageindex = self._index + (pageindex,)
5378                pages.append(
5379                    TiffFrame(
5380                        parent=page.parent,
5381                        index=pageindex,
5382                        offset=offset,
5383                        offsets=offsets,
5384                        bytecounts=page.databytecounts,
5385                        keyframe=page,
5386                    )
5387                )
5388            self.pages = pages
5389            self._cache = True
5390            self._cached = True
5391            self._indexed = True
5392        except Exception as exc:
5393            if self.parent.filehandle.size >= 2147483648:
5394                log_warning(f'TiffPages: failed to load virtual frames: {exc}')
5395
5396    def _clear(self, fully=True):
5397        """Delete all but first page from cache. Set keyframe to first page."""
5398        pages = self.pages
5399        if not pages:
5400            return
5401        self._keyframe = pages[0]
5402        if fully:
5403            # delete all but first TiffPage/TiffFrame
5404            for i, page in enumerate(pages[1:]):
5405                if not isinstance(page, int) and page.offset is not None:
5406                    pages[i + 1] = page.offset
5407        elif TiffFrame is not TiffPage:
5408            # delete only TiffFrames
5409            for i, page in enumerate(pages):
5410                if isinstance(page, TiffFrame) and page.offset is not None:
5411                    pages[i] = page.offset
5412        self._cached = False
5413
5414    def _seek(self, index, maxpages=None):
5415        """Seek file to offset of page specified by index."""
5416        pages = self.pages
5417        lenpages = len(pages)
5418        if lenpages == 0:
5419            raise IndexError('index out of range')
5420
5421        fh = self.parent.filehandle
5422        if fh.closed:
5423            raise ValueError('seek of closed file')
5424
5425        if self._indexed or 0 <= index < lenpages:
5426            page = pages[index]
5427            offset = page if isinstance(page, int) else page.offset
5428            fh.seek(offset)
5429            return
5430
5431        tiff = self.parent.tiff
5432        offsetformat = tiff.offsetformat
5433        offsetsize = tiff.offsetsize
5434        tagnoformat = tiff.tagnoformat
5435        tagnosize = tiff.tagnosize
5436        tagsize = tiff.tagsize
5437        unpack = struct.unpack
5438
5439        page = pages[-1]
5440        offset = page if isinstance(page, int) else page.offset
5441
5442        if maxpages is None:
5443            maxpages = 2 ** 22
5444        while lenpages < maxpages:
5445            # read offsets to pages from file until index is reached
5446            fh.seek(offset)
5447            # skip tags
5448            try:
5449                tagno = unpack(tagnoformat, fh.read(tagnosize))[0]
5450                if tagno > 4096:
5451                    raise TiffFileError(f'suspicious number of tags {tagno!r}')
5452            except Exception:
5453                log_warning(
5454                    'TiffPages: corrupted tag list of page '
5455                    f'{lenpages} @ {offset}'
5456                )
5457                del pages[-1]
5458                lenpages -= 1
5459                self._indexed = True
5460                break
5461            self._nextpageoffset = offset + tagnosize + tagno * tagsize
5462            fh.seek(self._nextpageoffset)
5463
5464            # read offset to next page
5465            offset = unpack(offsetformat, fh.read(offsetsize))[0]
5466            if offset == 0:
5467                self._indexed = True
5468                break
5469            if offset >= fh.size:
5470                log_warning(f'TiffPages: invalid page offset {offset!r}')
5471                self._indexed = True
5472                break
5473
5474            pages.append(offset)
5475            lenpages += 1
5476            if 0 <= index < lenpages:
5477                break
5478
5479            # detect some circular references
5480            if lenpages == 100:
5481                for p in pages[:-1]:
5482                    if offset == (p if isinstance(p, int) else p.offset):
5483                        raise TiffFileError('invalid circular IFD reference')
5484
5485        if index >= lenpages:
5486            raise IndexError('index out of range')
5487
5488        page = pages[index]
5489        fh.seek(page if isinstance(page, int) else page.offset)
5490
5491    def _getlist(self, key=None, useframes=True, validate=True):
5492        """Return specified pages as list of TiffPages or TiffFrames.
5493
5494        The first item is a TiffPage, and is used as a keyframe for
5495        following TiffFrames.
5496
5497        """
5498        getitem = self._getitem
5499        _useframes = self.useframes
5500
5501        if key is None:
5502            key = iter(range(len(self)))
5503        elif isinstance(key, Iterable):
5504            key = iter(key)
5505        elif isinstance(key, slice):
5506            start, stop, _ = key.indices(2 ** 31 - 1)
5507            if not self._indexed and max(stop, start) > len(self.pages):
5508                self._seek(-1)
5509            key = iter(range(*key.indices(len(self.pages))))
5510        elif isinstance(key, (int, numpy.integer)):
5511            # return single TiffPage
5512            self.useframes = False
5513            if key == 0:
5514                return [self.pages[key]]
5515            try:
5516                return [getitem(key)]
5517            finally:
5518                self.useframes = _useframes
5519        else:
5520            raise TypeError('key must be an integer, slice, or iterable')
5521
5522        # use first page as keyframe
5523        keyframe = self._keyframe
5524        self.keyframe = next(key)
5525        if validate:
5526            validate = self._keyframe.hash
5527        if useframes:
5528            self.useframes = True
5529        try:
5530            pages = [getitem(i, validate) for i in key]
5531            pages.insert(0, self._keyframe)
5532        finally:
5533            # restore state
5534            self._keyframe = keyframe
5535            if useframes:
5536                self.useframes = _useframes
5537
5538        return pages
5539
5540    def _getitem(self, key, validate=False, cache=None, aspage=None):
5541        """Return specified page from cache or file."""
5542        key = int(key)
5543        pages = self.pages
5544
5545        if key < 0:
5546            key %= len(self)
5547        elif self._indexed and key >= len(pages):
5548            raise IndexError(f'index {key} out of range({len(pages)})')
5549
5550        tiffpage = TiffPage if aspage else self._tiffpage
5551
5552        if key < len(pages):
5553            page = pages[key]
5554            if self._cache and not aspage:
5555                if not isinstance(page, (int, numpy.integer)):
5556                    if validate and validate != page.hash:
5557                        raise RuntimeError('page hash mismatch')
5558                    return page
5559            elif isinstance(page, (TiffPage, tiffpage)):
5560                if validate and validate != page.hash:
5561                    raise RuntimeError('page hash mismatch')
5562                return page
5563
5564        pageindex = key if self._index is None else self._index + (key,)
5565        self._seek(key)
5566        page = tiffpage(self.parent, index=pageindex, keyframe=self._keyframe)
5567        if validate and validate != page.hash:
5568            raise RuntimeError('page hash mismatch')
5569        if self._cache or cache:
5570            pages[key] = page
5571        return page
5572
5573    def __getitem__(self, key):
5574        """Return specified page(s)."""
5575        pages = self.pages
5576        getitem = self._getitem
5577
5578        if isinstance(key, (int, numpy.integer)):
5579            if key == 0:
5580                return pages[key]
5581            return getitem(key)
5582
5583        if isinstance(key, slice):
5584            start, stop, _ = key.indices(2 ** 31 - 1)
5585            if not self._indexed and max(stop, start) > len(pages):
5586                self._seek(-1)
5587            return [getitem(i) for i in range(*key.indices(len(pages)))]
5588
5589        if isinstance(key, Iterable):
5590            return [getitem(k) for k in key]
5591
5592        raise TypeError('key must be an integer, slice, or iterable')
5593
5594    def __iter__(self):
5595        """Return iterator over all pages."""
5596        i = 0
5597        while True:
5598            try:
5599                yield self._getitem(i)
5600                i += 1
5601            except IndexError:
5602                break
5603        if self._cache:
5604            self._cached = True
5605
5606    def __bool__(self):
5607        """Return True if file contains any pages."""
5608        return len(self.pages) > 0
5609
5610    def __len__(self):
5611        """Return number of pages in file."""
5612        if not self._indexed:
5613            self._seek(-1)
5614        return len(self.pages)
5615
5616
5617class TiffPage:
5618    """TIFF image file directory (IFD).
5619
5620    Attributes
5621    ----------
5622    index : int
5623        Index of the page in file.
5624    dtype : numpy.dtype or None
5625        Data type (native byte order) of the image in IFD.
5626    shape : tuple of int
5627        Dimensions of the image in IFD, as returned by asarray.
5628    axes : str
5629        Axes label codes for each dimension in shape:
5630        'S' sample,
5631        'X' width,
5632        'Y' length,
5633        'Z' depth,
5634    tags : TiffTags
5635        Multidict like interface to tags in IFD.
5636    colormap : numpy.ndarray
5637        Color look up table, if exists.
5638    shaped : tuple of int
5639        Normalized 5-dimensional shape of the image in IFD:
5640        0 : separate samplesperpixel or 1.
5641        1 : imagedepth Z or 1.
5642        2 : imagelength Y.
5643        3 : imagewidth X.
5644        4 : contig samplesperpixel or 1.
5645
5646    All attributes are read-only.
5647
5648    """
5649
5650    # default properties; will be updated from tags
5651    subfiletype = 0
5652    imagewidth = 0
5653    imagelength = 0
5654    imagedepth = 1
5655    tilewidth = 0
5656    tilelength = 0
5657    tiledepth = 1
5658    bitspersample = 1
5659    samplesperpixel = 1
5660    sampleformat = 1
5661    rowsperstrip = 2 ** 32 - 1
5662    compression = 1
5663    planarconfig = 1
5664    fillorder = 1
5665    photometric = 0
5666    predictor = 1
5667    extrasamples = ()
5668    subsampling = None
5669    subifds = None
5670    jpegtables = None
5671    jpegheader = None  # NDPI only
5672    colormap = None
5673    software = ''
5674    description = ''
5675    description1 = ''
5676    nodata = 0
5677
5678    def __init__(self, parent, index, keyframe=None):
5679        """Initialize instance from file.
5680
5681        The file handle position must be at offset to a valid IFD.
5682
5683        """
5684        self.parent = parent
5685        self.index = index
5686        self.shape = ()
5687        self.shaped = ()
5688        self.dtype = None
5689        self._dtype = None
5690        self.axes = ''
5691        self.tags = tags = TiffTags()
5692        self.dataoffsets = ()
5693        self.databytecounts = ()
5694
5695        tiff = parent.tiff
5696
5697        # read TIFF IFD structure and its tags from file
5698        fh = parent.filehandle
5699        self.offset = fh.tell()  # offset to this IFD
5700        try:
5701            tagno = struct.unpack(tiff.tagnoformat, fh.read(tiff.tagnosize))[0]
5702            if tagno > 4096:
5703                raise ValueError(f'suspicious number of tags {tagno}')
5704        except Exception as exc:
5705            raise TiffFileError(
5706                f'TiffPage {self.index}: '
5707                f'corrupted tag list at offset {self.offset}'
5708            ) from exc
5709
5710        tagoffset = self.offset + tiff.tagnosize  # fh.tell()
5711        tagsize = tagsize_ = tiff.tagsize
5712
5713        data = fh.read(tagsize * tagno)
5714
5715        if tiff.version == 42 and tiff.offsetsize == 8:
5716            # patch offsets/values for 64-bit NDPI file
5717            tagsize = 16
5718            fh.seek(8, os.SEEK_CUR)
5719            ext = fh.read(4 * tagno)  # high bits
5720            data = b''.join(
5721                data[i * 12 : i * 12 + 12] + ext[i * 4 : i * 4 + 4]
5722                for i in range(tagno)
5723            )
5724
5725        tagindex = -tagsize
5726        for i in range(tagno):
5727            tagindex += tagsize
5728            tagdata = data[tagindex : tagindex + tagsize]
5729            try:
5730                tag = TiffTag.fromfile(
5731                    parent, tagoffset + i * tagsize_, tagdata
5732                )
5733            except TiffFileError as exc:
5734                log_warning(
5735                    f'TiffPage {self.index}: {exc.__class__.__name__}: {exc}'
5736                )
5737                continue
5738            tags.add(tag)
5739
5740        if not tags:
5741            return  # found in FIBICS
5742
5743        for code, name in TIFF.TAG_ATTRIBUTES.items():
5744            tag = tags.get(code)
5745            if tag is not None:
5746                if code in (270, 305) and not isinstance(tag.value, str):
5747                    # wrong string type for software or description
5748                    continue
5749                setattr(self, name, tag.value)
5750
5751        tag = tags.get(270, index=1)
5752        if tag:
5753            self.description1 = tag.value
5754
5755        if self.subfiletype == 0:
5756            tag = tags.get(255)  # SubfileType
5757            if tag:
5758                if tag.value == 2:
5759                    self.subfiletype = 0b1  # reduced image
5760                elif tag.value == 3:
5761                    self.subfiletype = 0b10  # multi-page
5762
5763        # consolidate private tags; remove them from self.tags
5764        # if self.is_andor:
5765        #     self.andor_tags
5766        # elif self.is_epics:
5767        #     self.epics_tags
5768        # elif self.is_ndpi:
5769        #     self.ndpi_tags
5770        # if self.is_sis and 34853 in tags:
5771        #     # TODO: can't change tag.name
5772        #     tags[34853].name = 'OlympusSIS2'
5773
5774        # dataoffsets and databytecounts
5775        if 324 in tags:  # TileOffsets
5776            self.dataoffsets = tags[324].value
5777        elif 273 in tags:  # StripOffsets
5778            self.dataoffsets = tags[273].value
5779        elif 513 in tags:  # JPEGInterchangeFormat et al.
5780            self.dataoffsets = tags[513].value
5781        if 325 in tags:  # TileByteCounts
5782            self.databytecounts = tags[325].value
5783        elif 279 in tags:  # StripByteCounts
5784            self.databytecounts = tags[279].value
5785        elif 514 in tags:  # JPEGInterchangeFormatLength et al.
5786            self.databytecounts = tags[514].value
5787
5788        if (
5789            self.imagewidth == 0
5790            and self.imagelength == 0
5791            and self.dataoffsets
5792            and self.databytecounts
5793        ):
5794            # dimensions may be missing in some RAW formats
5795            # read dimensions from assumed JPEG encoded segment
5796            try:
5797                fh.seek(self.dataoffsets[0])
5798                (
5799                    precision,
5800                    imagelength,
5801                    imagewidth,
5802                    samplesperpixel,
5803                ) = jpeg_shape(fh.read(min(self.databytecounts[0], 4096)))
5804            except Exception:
5805                pass
5806            else:
5807                self.imagelength = imagelength
5808                self.imagewidth = imagewidth
5809                self.samplesperpixel = samplesperpixel
5810                if 258 not in tags:
5811                    self.bitspersample = 8 if precision <= 8 else 16
5812                if 262 not in tags and samplesperpixel == 3:
5813                    self.photometric = 6  # YCbCr
5814                if 259 not in tags:
5815                    self.compression = 6  # OJPEG
5816                if 278 not in tags:
5817                    self.rowsperstrip = imagelength
5818
5819        elif self.compression == 6:
5820            # OJPEG hack. See libtiff v4.2.0 tif_dirread.c#L4082
5821            if 262 not in tags:
5822                # PhotometricInterpretation missing
5823                self.photometric = 6  # YCbCr
5824            elif self.photometric == 2:
5825                # RGB -> YCbCr
5826                self.photometric = 6
5827            if 258 not in tags:
5828                # BitsPerSample missing
5829                self.bitspersample = 8
5830            if 277 not in tags:
5831                # SamplesPerPixel missing
5832                if self.photometric in (2, 6):
5833                    self.samplesperpixel = 3
5834                elif self.photometric in (0, 1):
5835                    self.samplesperpixel = 3
5836
5837        elif self.is_lsm or (self.index != 0 and self.parent.is_lsm):
5838            # correct non standard LSM bitspersample tags
5839            tags[258]._fix_lsm_bitspersample()
5840            if self.compression == 1 and self.predictor != 1:
5841                # work around bug in LSM510 software
5842                self.predictor = 1
5843
5844        elif self.is_vista or (self.index != 0 and self.parent.is_vista):
5845            # ISS Vista writes wrong ImageDepth tag
5846            self.imagedepth = 1
5847
5848        elif self.is_stk:
5849            tag = tags.get(33628)  # UIC1tag
5850            if tag is not None and not tag.value:
5851                # read UIC1tag now that plane count is known
5852                fh.seek(tag.valueoffset)
5853                tag.value = read_uic1tag(
5854                    fh,
5855                    tiff.byteorder,
5856                    tag.dtype,
5857                    tag.count,
5858                    None,
5859                    tags[33629].count,  # UIC2tag
5860                )
5861
5862        if 50839 in tags:
5863            # decode IJMetadata tag
5864            try:
5865                tags[50839].value = imagej_metadata(
5866                    tags[50839].value,
5867                    tags[50838].value,  # IJMetadataByteCounts
5868                    tiff.byteorder,
5869                )
5870            except Exception as exc:
5871                log_warning(
5872                    f'TiffPage {self.index}: {exc.__class__.__name__}: {exc}'
5873                )
5874
5875        # BitsPerSample
5876        tag = tags.get(258)
5877        if tag is not None:
5878            if self.bitspersample != 1:
5879                pass  # bitspersample was set by ojpeg hack
5880            elif tag.count == 1:
5881                self.bitspersample = tag.value
5882            else:
5883                # LSM might list more items than samplesperpixel
5884                value = tag.value[: self.samplesperpixel]
5885                if any(v - value[0] for v in value):
5886                    self.bitspersample = value
5887                else:
5888                    self.bitspersample = value[0]
5889
5890        # SampleFormat
5891        tag = tags.get(339)
5892        if tag is not None:
5893            if tag.count == 1:
5894                self.sampleformat = tag.value
5895            else:
5896                value = tag.value[: self.samplesperpixel]
5897                if any(v - value[0] for v in value):
5898                    self.sampleformat = value
5899                else:
5900                    self.sampleformat = value[0]
5901
5902        if 322 in tags:  # TileWidth
5903            self.rowsperstrip = None
5904        elif 257 in tags:  # ImageLength
5905            if 278 not in tags or tags[278].count > 1:  # RowsPerStrip
5906                self.rowsperstrip = self.imagelength
5907            self.rowsperstrip = min(self.rowsperstrip, self.imagelength)
5908            # self.stripsperimage = int(math.floor(
5909            #    float(self.imagelength + self.rowsperstrip - 1) /
5910            #    self.rowsperstrip))
5911
5912        # determine dtype
5913        dtype = TIFF.SAMPLE_DTYPES.get(
5914            (self.sampleformat, self.bitspersample), None
5915        )
5916        if dtype is not None:
5917            dtype = numpy.dtype(dtype)
5918        self.dtype = self._dtype = dtype
5919
5920        # determine shape of data
5921        imagelength = self.imagelength
5922        imagewidth = self.imagewidth
5923        imagedepth = self.imagedepth
5924        samplesperpixel = self.samplesperpixel
5925
5926        if self.photometric == 2 or samplesperpixel > 1:  # PHOTOMETRIC.RGB
5927            if self.planarconfig == 1:
5928                self.shaped = (
5929                    1,
5930                    imagedepth,
5931                    imagelength,
5932                    imagewidth,
5933                    samplesperpixel,
5934                )
5935                if imagedepth == 1:
5936                    self.shape = (imagelength, imagewidth, samplesperpixel)
5937                    self.axes = 'YXS'
5938                else:
5939                    self.shape = (
5940                        imagedepth,
5941                        imagelength,
5942                        imagewidth,
5943                        samplesperpixel,
5944                    )
5945                    self.axes = 'ZYXS'
5946            else:
5947                self.shaped = (
5948                    samplesperpixel,
5949                    imagedepth,
5950                    imagelength,
5951                    imagewidth,
5952                    1,
5953                )
5954                if imagedepth == 1:
5955                    self.shape = (samplesperpixel, imagelength, imagewidth)
5956                    self.axes = 'SYX'
5957                else:
5958                    self.shape = (
5959                        samplesperpixel,
5960                        imagedepth,
5961                        imagelength,
5962                        imagewidth,
5963                    )
5964                    self.axes = 'SZYX'
5965        else:
5966            self.shaped = (1, imagedepth, imagelength, imagewidth, 1)
5967            if imagedepth == 1:
5968                self.shape = (imagelength, imagewidth)
5969                self.axes = 'YX'
5970            else:
5971                self.shape = (imagedepth, imagelength, imagewidth)
5972                self.axes = 'ZYX'
5973
5974        if not self.databytecounts:
5975            self.databytecounts = (
5976                product(self.shape) * (self.bitspersample // 8),
5977            )
5978            if self.compression != 1:
5979                log_warning(
5980                    f'TiffPage {self.index}: ByteCounts tag is missing'
5981                )
5982
5983        if imagelength and self.rowsperstrip and not self.is_lsm:
5984            # fix incorrect number of strip bytecounts and offsets
5985            maxstrips = (
5986                int(
5987                    math.floor(imagelength + self.rowsperstrip - 1)
5988                    / self.rowsperstrip
5989                )
5990                * self.imagedepth
5991            )
5992            if self.planarconfig == 2:
5993                maxstrips *= self.samplesperpixel
5994            if maxstrips != len(self.databytecounts):
5995                log_warning(
5996                    f'TiffPage {self.index}: incorrect StripByteCounts count'
5997                )
5998                self.databytecounts = self.databytecounts[:maxstrips]
5999            if maxstrips != len(self.dataoffsets):
6000                log_warning(
6001                    f'TiffPage {self.index}: incorrect StripOffsets count'
6002                )
6003                self.dataoffsets = self.dataoffsets[:maxstrips]
6004
6005        tag = tags.get(42113)  # GDAL_NODATA
6006        if tag is not None:
6007            try:
6008                pytype = type(dtype.type(0).item())
6009                self.nodata = pytype(tag.value)
6010            except Exception:
6011                pass
6012
6013        if 65426 in tags and self.is_ndpi:
6014            # use NDPI JPEG McuStarts as tile offsets
6015            mcustarts = tags[65426].value
6016            if 65432 in tags:
6017                # McuStartsHighBytes
6018                high = tags[65432].value.astype('uint64')
6019                high <<= 32
6020                mcustarts = mcustarts.astype('uint64')
6021                mcustarts += high
6022            fh.seek(self.dataoffsets[0])
6023            jpegheader = fh.read(mcustarts[0])
6024            try:
6025                (
6026                    self.tilelength,
6027                    self.tilewidth,
6028                    self.jpegheader,
6029                ) = ndpi_jpeg_tile(jpegheader)
6030            except ValueError as exc:
6031                log_warning(
6032                    f'TiffPage {self.index}: ndpi_jpeg_tile failed ({exc})'
6033                )
6034            else:
6035                self.databytecounts = (
6036                    mcustarts[1:] - mcustarts[:-1]
6037                ).tolist() + [self.databytecounts[0] - int(mcustarts[-1])]
6038                self.dataoffsets = (mcustarts + self.dataoffsets[0]).tolist()
6039
6040    @lazyattr
6041    def decode(self):
6042        """Return decoded segment, its shape, and indices in image.
6043
6044        The decode function is implemeted as a closure.
6045
6046        Parameters
6047        ----------
6048        data : bytes
6049            Encoded bytes of a segment (aka strile, strip or tile)
6050            or None for empty segments.
6051        index : int
6052            The index of the segment in the Offsets and Bytecount tag values.
6053        jpegtables : bytes or None
6054            For JPEG compressed segments only, the value of the JPEGTables tag
6055            if any.
6056
6057        Returns
6058        -------
6059        segment : numpy.ndarray
6060            Decoded segment or None for empty segments.
6061        indices : tuple of int
6062            The position of the segment in the image array of normalized shape:
6063            (separate sample, depth, length, width, contig sample).
6064        shape : tuple of int
6065            The shape of the segment: (depth, length, width, contig samples).
6066            The shape of strips depends on their linear index.
6067
6068        Raises ValueError or NotImplementedError if decoding is not supported.
6069
6070        """
6071        if self.hash in self.parent._parent._decoders:
6072            return self.parent._parent._decoders[self.hash]
6073
6074        def cache(decode):
6075            self.parent._parent._decoders[self.hash] = decode
6076            return decode
6077
6078        if self.dtype is None:
6079
6080            def decode(*args, **kwargs):
6081                raise ValueError(
6082                    f'TiffPage {self.index}: data type not supported: '
6083                    f'SampleFormat {self.sampleformat}, '
6084                    f'{self.bitspersample}-bit'
6085                )
6086
6087            return cache(decode)
6088
6089        if 0 in self.shaped:
6090
6091            def decode(*args, **kwargs):
6092                raise ValueError(f'TiffPage {self.index}: empty image')
6093
6094            return cache(decode)
6095
6096        try:
6097            if self.compression == 1:
6098                decompress = None
6099            else:
6100                decompress = TIFF.DECOMPRESSORS[self.compression]
6101        except KeyError as exc:
6102
6103            def decode(*args, exc=str(exc)[1:-1], **kwargs):
6104                raise ValueError(f'TiffPage {self.index}: {exc}')
6105
6106            return cache(decode)
6107
6108        try:
6109            if self.predictor == 1:
6110                unpredict = None
6111            else:
6112                unpredict = TIFF.UNPREDICTORS[self.predictor]
6113        except KeyError as exc:
6114
6115            def decode(*args, exc=str(exc)[1:-1], **kwargs):
6116                raise ValueError(f'TiffPage {self.index}: {exc}')
6117
6118            return cache(decode)
6119
6120        if self.tags.get(339) is not None:
6121            tag = self.tags[339]  # SampleFormat
6122            if tag.count != 1 and any(i - tag.value[0] for i in tag.value):
6123
6124                def decode(*args, **kwargs):
6125                    raise ValueError(
6126                        f'TiffPage {self.index}: '
6127                        f'sample formats do not match {tag.value}'
6128                    )
6129
6130                return cache(decode)
6131
6132        if self.is_subsampled and (
6133            self.compression not in (6, 7) or self.planarconfig == 2
6134        ):
6135
6136            def decode(*args, **kwargs):
6137                raise NotImplementedError(
6138                    f'TiffPage {self.index}: chroma subsampling not supported'
6139                )
6140
6141            return cache(decode)
6142
6143        # normalize segments shape to [depth, length, length, contig]
6144        if self.is_tiled:
6145            stshape = [self.tiledepth, self.tilelength, self.tilewidth, 1]
6146        else:
6147            stshape = [1, self.rowsperstrip, self.imagewidth, 1]
6148        if self.planarconfig == 1:
6149            stshape[-1] = self.samplesperpixel
6150        stshape = tuple(stshape)
6151
6152        stdepth, stlength, stwidth, samples = stshape
6153        imdepth, imlength, imwidth, samples = self.shaped[1:]
6154
6155        if self.is_tiled:
6156
6157            width = (imwidth + stwidth - 1) // stwidth
6158            length = (imlength + stlength - 1) // stlength
6159            depth = (imdepth + stdepth - 1) // stdepth
6160
6161            def indices(tileindex):
6162                # return indices and shape of tile in image array
6163                return (
6164                    (
6165                        tileindex // (width * length * depth),
6166                        (tileindex // (width * length)) % depth * stdepth,
6167                        (tileindex // width) % length * stlength,
6168                        tileindex % width * stwidth,
6169                        0,
6170                    ),
6171                    stshape,
6172                )
6173
6174            def reshape(data, indices, shape):
6175                # return reshaped tile
6176                if data is None:
6177                    return data
6178                size = shape[0] * shape[1] * shape[2] * shape[3]
6179                if data.size > size:
6180                    # decompression / unpacking might return too many bytes
6181                    data.shape = -1
6182                    data = data[:size]
6183                if data.size == size:
6184                    # complete tile
6185                    # data might be non-contiguous; cannot reshape inplace
6186                    data = data.reshape(shape)
6187                else:
6188                    # data fills remaining space
6189                    # found in some JPEG/PNG compressed tiles
6190                    try:
6191                        data = data.reshape(
6192                            (
6193                                min(imdepth - indices[1], shape[0]),
6194                                min(imlength - indices[2], shape[1]),
6195                                min(imwidth - indices[3], shape[2]),
6196                                samples,
6197                            )
6198                        )
6199                    except ValueError:
6200                        # incomplete tile; see gdal issue #1179
6201                        log_warning(
6202                            f'reshape: incomplete tile {data.shape} {shape}'
6203                        )
6204                        t = numpy.zeros(size, data.dtype)
6205                        size = min(data.size, size)
6206                        t[:size] = data[:size]
6207                        data = t.reshape(shape)
6208                return data
6209
6210            def pad(data, shape, nodata=self.nodata):
6211                # pad tile to shape
6212                if data is None or data.shape == shape:
6213                    return data, shape
6214                padwidth = [(0, i - j) for i, j in zip(shape, data.shape)]
6215                data = numpy.pad(data, padwidth, constant_values=nodata)
6216                return data, data.shape
6217
6218        else:
6219            # strips
6220            length = (imlength + stlength - 1) // stlength
6221
6222            def indices(stripindex):
6223                # return indices and shape of strip in image array
6224                indices = (
6225                    stripindex // (length * imdepth),
6226                    (stripindex // length) % imdepth * stdepth,
6227                    stripindex % length * stlength,
6228                    0,
6229                    0,
6230                )
6231                shape = (
6232                    stdepth,
6233                    min(stlength, imlength - indices[2]),
6234                    stwidth,
6235                    samples,
6236                )
6237                return indices, shape
6238
6239            def reshape(data, indices, shape):
6240                # return reshaped strip
6241                if data is None:
6242                    return data
6243                size = shape[0] * shape[1] * shape[2] * shape[3]
6244                if data.size > size:
6245                    # decompression / unpacking might return too many bytes
6246                    data.shape = -1
6247                    data = data[:size]
6248                if data.size == size:
6249                    # expected size
6250                    data.shape = shape
6251                else:
6252                    # should not happen, but try different length
6253                    data.shape = shape[0], -1, shape[2], shape[3]
6254                    # raise RuntimeError(
6255                    #     f'invalid strip shape {data.shape} or size {size}'
6256                    # )
6257                return data
6258
6259            def pad(data, shape, nodata=self.nodata):
6260                # pad strip length to rowsperstrip
6261                shape = (shape[0], stlength, shape[2], shape[3])
6262                if data is None or data.shape == shape:
6263                    return data, shape
6264                padwidth = [
6265                    (0, 0),
6266                    (0, stlength - data.shape[1]),
6267                    (0, 0),
6268                    (0, 0),
6269                ]
6270                data = numpy.pad(data, padwidth, constant_values=nodata)
6271                return data, data.shape
6272
6273        if self.compression in (6, 7, 34892, 33007):
6274            # JPEG needs special handling
6275            if self.fillorder == 2:
6276                log_warning(
6277                    f'TiffPage {self.index}: disabling LSB2MSB for JPEG'
6278                )
6279            if unpredict:
6280                log_warning(
6281                    f'TiffPage {self.index}: disabling predictor for JPEG'
6282                )
6283            if 28672 in self.tags:  # SonyRawFileType
6284                log_warning(
6285                    f'TiffPage {self.index}: '
6286                    'SonyRawFileType might need additional unpacking '
6287                    '(see issue #95)'
6288                )
6289
6290            colorspace, outcolorspace = jpeg_decode_colorspace(
6291                self.photometric, self.planarconfig, self.extrasamples
6292            )
6293
6294            def decode(
6295                data,
6296                segmentindex,
6297                jpegtables=None,
6298                jpegheader=None,
6299                _fullsize=False,
6300                bitspersample=self.bitspersample,
6301                colorspace=colorspace,
6302                outcolorspace=outcolorspace,
6303            ):
6304                # return decoded segment, its shape, and indices in image
6305                index, shape = indices(segmentindex)
6306                if data is None:
6307                    if _fullsize:
6308                        data, shape = pad(data, shape)
6309                    return data, index, shape
6310                data = imagecodecs.jpeg_decode(
6311                    data,
6312                    bitspersample=bitspersample,
6313                    tables=jpegtables,
6314                    header=jpegheader,
6315                    colorspace=colorspace,
6316                    outcolorspace=outcolorspace,
6317                    shape=shape[1:3],
6318                )
6319                data = reshape(data, index, shape)
6320                if _fullsize:
6321                    data, shape = pad(data, shape)
6322                return data, index, shape
6323
6324            return cache(decode)
6325
6326        if self.compression in (
6327            33003,
6328            33004,
6329            33005,
6330            34712,
6331            34933,
6332            34934,
6333            22610,
6334            50001,
6335            50002,
6336        ):
6337            # JPEG2000, WEBP, PNG, JPEGXR
6338            # presume codecs always return correct dtype, native byte order...
6339            if self.fillorder == 2:
6340                log_warning(
6341                    f'TiffPage {self.index}: disabling LSB2MSB for '
6342                    f'compression {self.compression}'
6343                )
6344            if unpredict:
6345                log_warning(
6346                    f'TiffPage {self.index}: disabling predictor for '
6347                    f'compression {self.compression}'
6348                )
6349
6350            def decode(data, segmentindex, jpegtables=None, _fullsize=False):
6351                # return decoded segment, its shape, and indices in image
6352                index, shape = indices(segmentindex)
6353                if data is None:
6354                    if _fullsize:
6355                        data, shape = pad(data, shape)
6356                    return data, index, shape
6357                data = decompress(data)
6358                data = reshape(data, index, shape)
6359                if _fullsize:
6360                    data, shape = pad(data, shape)
6361                return data, index, shape
6362
6363            return cache(decode)
6364
6365        dtype = numpy.dtype(self.parent.byteorder + self._dtype.char)
6366
6367        if self.sampleformat == 5:
6368            # complex integer
6369            if unpredict is not None:
6370                raise NotImplementedError(
6371                    'unpredicting complex integers not supported'
6372                )
6373
6374            itype = numpy.dtype(
6375                f'{self.parent.byteorder}i{self.bitspersample // 16}'
6376            )
6377            ftype = numpy.dtype(
6378                f'{self.parent.byteorder}f{dtype.itemsize // 2}'
6379            )
6380
6381            def unpack(data):
6382                # return complex integer as numpy.complex
6383                data = numpy.frombuffer(data, itype)
6384                return data.astype(ftype).view(dtype)
6385
6386        elif self.bitspersample in (8, 16, 32, 64, 128):
6387            # regular data types
6388
6389            if (self.bitspersample * stwidth * samples) % 8:
6390                raise ValueError(
6391                    f'TiffPage {self.index}: data and sample size mismatch'
6392                )
6393            if self.predictor == 3:  # PREDICTOR.FLOATINGPOINT
6394                # floating-point horizontal differencing decoder needs
6395                # raw byte order
6396                dtype = numpy.dtype(self._dtype.char)
6397
6398            def unpack(data):
6399                # return numpy array from buffer
6400                try:
6401                    # read only numpy array
6402                    return numpy.frombuffer(data, dtype)
6403                except ValueError:
6404                    # e.g. LZW strips may be missing EOI
6405                    bps = self.bitspersample // 8
6406                    size = (len(data) // bps) * bps
6407                    return numpy.frombuffer(data[:size], dtype)
6408
6409        elif isinstance(self.bitspersample, tuple):
6410            # e.g. RGB 565
6411            def unpack(data):
6412                # return numpy array from packed integers
6413                return unpack_rgb(data, dtype, self.bitspersample)
6414
6415        elif self.bitspersample == 24 and dtype.char == 'f':
6416            # float24
6417            if unpredict is not None:
6418                # floatpred_decode requires numpy.float24, which does not exist
6419                raise NotImplementedError('unpredicting float24 not supported')
6420
6421            def unpack(data, byteorder=self.parent.byteorder):
6422                # return numpy.float32 array from float24
6423                return float24_decode(data, byteorder)
6424
6425        else:
6426            # bilevel and packed integers
6427            def unpack(data):
6428                # return numpy array from packed integers
6429                return packints_decode(
6430                    data, dtype, self.bitspersample, stwidth * samples
6431                )
6432
6433        def decode(data, segmentindex, jpegtables=None, _fullsize=False):
6434            # return decoded segment, its shape, and indices in image
6435            index, shape = indices(segmentindex)
6436            if data is None:
6437                if _fullsize:
6438                    data, shape = pad(data, shape)
6439                return data, index, shape
6440            if self.fillorder == 2:
6441                data = bitorder_decode(data)
6442            if decompress is not None:
6443                # TODO: calculate correct size for packed integers
6444                size = shape[0] * shape[1] * shape[2] * shape[3]
6445                data = decompress(data, out=size * dtype.itemsize)
6446            data = unpack(data)
6447            data = reshape(data, index, shape)
6448            data = data.astype('=' + dtype.char)
6449            if unpredict is not None:
6450                # unpredict is faster with native byte order
6451                data = unpredict(data, axis=-2, out=data)
6452            if _fullsize:
6453                data, shape = pad(data, shape)
6454            return data, index, shape
6455
6456        return cache(decode)
6457
6458    def segments(
6459        self, lock=None, maxworkers=None, func=None, sort=False, _fullsize=None
6460    ):
6461        """Return iterator over decoded segments in TiffPage.
6462
6463        See the decode function for return values.
6464
6465        """
6466        keyframe = self.keyframe  # self or keyframe
6467        fh = self.parent.filehandle
6468        if lock is None:
6469            lock = fh.lock
6470        if _fullsize is None:
6471            _fullsize = keyframe.is_tiled
6472
6473        decodeargs = {'_fullsize': bool(_fullsize)}
6474        if keyframe.compression in (6, 7, 34892, 33007):  # JPEG
6475            decodeargs['jpegtables'] = self.jpegtables
6476            decodeargs['jpegheader'] = keyframe.jpegheader
6477
6478        if func is None:
6479
6480            def decode(args, decodeargs=decodeargs, keyframe=keyframe):
6481                return keyframe.decode(*args, **decodeargs)
6482
6483        else:
6484
6485            def decode(
6486                args, decodeargs=decodeargs, keyframe=keyframe, func=func
6487            ):
6488                return func(keyframe.decode(*args, **decodeargs))
6489
6490        if maxworkers is None or maxworkers < 1:
6491            maxworkers = keyframe.maxworkers
6492        if maxworkers < 2:
6493            for segment in fh.read_segments(
6494                self.dataoffsets,
6495                self.databytecounts,
6496                lock=lock,
6497                sort=sort,
6498                flat=True,
6499            ):
6500                yield decode(segment)
6501        else:
6502            # reduce memory overhead by processing chunks of up to
6503            # ~64 MB of segments because ThreadPoolExecutor.map is not
6504            # collecting iterables lazily
6505            with ThreadPoolExecutor(maxworkers) as executor:
6506                for segments in fh.read_segments(
6507                    self.dataoffsets,
6508                    self.databytecounts,
6509                    lock=lock,
6510                    sort=sort,
6511                    flat=False,
6512                ):
6513                    yield from executor.map(decode, segments)
6514
6515    def asarray(self, out=None, squeeze=True, lock=None, maxworkers=None):
6516        """Read image data from file and return as numpy array.
6517
6518        Raise ValueError if format is not supported.
6519
6520        Parameters
6521        ----------
6522        out : numpy.ndarray, str, or file-like object
6523            Buffer where image data are saved.
6524            If None (default), a new array is created.
6525            If numpy.ndarray, a writable array of compatible dtype and shape.
6526            If 'memmap', directly memory-map the image data in the TIFF file
6527            if possible; else create a memory-mapped array in a temporary file.
6528            If str or open file, the file name or file object used to
6529            create a memory-map to an array stored in a binary file on disk.
6530        squeeze : bool
6531            If True (default), all length-1 dimensions (except X and Y) are
6532            squeezed out from the array.
6533            If False, the shape of the returned array is the normalized
6534            5-dimensional shape (TiffPage.shaped).
6535        lock : {RLock, NullContext}
6536            A reentrant lock used to synchronize seeks and reads from file.
6537            If None (default), the lock of the parent's filehandle is used.
6538        maxworkers : int or None
6539            Maximum number of threads to concurrently decode strips ot tiles.
6540            If None (default), up to half the CPU cores are used.
6541            See remarks in TiffFile.asarray.
6542
6543        Returns
6544        -------
6545        numpy.ndarray
6546            Numpy array of decompressed, unpredicted, and unpacked image data
6547            read from Strip/Tile Offsets/ByteCounts, formatted according to
6548            shape and dtype metadata found in tags and parameters.
6549            Photometric conversion, pre-multiplied alpha, orientation, and
6550            colorimetry corrections are not applied. Specifically, CMYK images
6551            are not converted to RGB, MinIsWhite images are not inverted,
6552            and color palettes are not applied. Exception are YCbCr JPEG
6553            compressed images, which are converted to RGB.
6554
6555        """
6556        keyframe = self.keyframe  # self or keyframe
6557
6558        if not keyframe.shaped or product(keyframe.shaped) == 0:
6559            return None
6560
6561        fh = self.parent.filehandle
6562        if lock is None:
6563            lock = fh.lock
6564        with lock:
6565            closed = fh.closed
6566            if closed:
6567                fh.open()
6568
6569        if (
6570            isinstance(out, str)
6571            and out == 'memmap'
6572            and keyframe.is_memmappable
6573        ):
6574            # direct memory map array in file
6575            with lock:
6576                result = fh.memmap_array(
6577                    keyframe.parent.byteorder + keyframe._dtype.char,
6578                    keyframe.shaped,
6579                    offset=self.dataoffsets[0],
6580                )
6581
6582        elif keyframe.is_contiguous:
6583            # read contiguous bytes to array
6584            if keyframe.is_subsampled:
6585                raise NotImplementedError(
6586                    f'TiffPage {self.index}: chroma subsampling not supported'
6587                )
6588            if out is not None:
6589                out = create_output(out, keyframe.shaped, keyframe._dtype)
6590            with lock:
6591                fh.seek(self.dataoffsets[0])
6592                result = fh.read_array(
6593                    keyframe.parent.byteorder + keyframe._dtype.char,
6594                    product(keyframe.shaped),
6595                    out=out,
6596                )
6597            if keyframe.fillorder == 2:
6598                bitorder_decode(result, out=result)
6599            if keyframe.predictor != 1:
6600                # predictors without compression
6601                unpredict = TIFF.UNPREDICTORS[keyframe.predictor]
6602                if keyframe.predictor == 1:
6603                    unpredict(result, axis=-2, out=result)
6604                else:
6605                    # floatpred cannot decode in-place
6606                    out = unpredict(result, axis=-2, out=result)
6607                    result[:] = out
6608
6609        elif (
6610            keyframe.jpegheader is not None
6611            and keyframe is self
6612            and 273 in self.tags  # striped ...
6613            and self.is_tiled  # but reported as tiled
6614            and self.imagewidth <= 65500
6615            and self.imagelength <= 65500
6616        ):
6617            # decode the whole NDPI JPEG strip
6618            with lock:
6619                fh.seek(self.tags[273].value[0])  # StripOffsets
6620                data = fh.read(self.tags[279].value[0])  # StripByteCounts
6621            decompress = TIFF.DECOMPRESSORS[self.compression]
6622            result = decompress(
6623                data, bitspersample=self.bitspersample, out=out
6624            )
6625            del data
6626
6627        else:
6628            # decode individual strips or tiles
6629            result = create_output(out, keyframe.shaped, keyframe._dtype)
6630            keyframe.decode  # init TiffPage.decode function
6631
6632            def func(decoderesult, keyframe=keyframe, out=out):
6633                # copy decoded segments to output array
6634                segment, (s, d, l, w, _), shape = decoderesult
6635                if segment is None:
6636                    segment = keyframe.nodata
6637                else:
6638                    segment = segment[
6639                        : keyframe.imagedepth - d,
6640                        : keyframe.imagelength - l,
6641                        : keyframe.imagewidth - w,
6642                    ]
6643                result[
6644                    s, d : d + shape[0], l : l + shape[1], w : w + shape[2]
6645                ] = segment
6646                # except IndexError:
6647                #     pass  # corrupted files e.g. with too many strips
6648
6649            for _ in self.segments(
6650                func=func,
6651                lock=lock,
6652                maxworkers=maxworkers,
6653                sort=True,
6654                _fullsize=False,
6655            ):
6656                pass
6657
6658        result.shape = keyframe.shaped
6659        if squeeze:
6660            try:
6661                result.shape = keyframe.shape
6662            except ValueError:
6663                log_warning(
6664                    f'TiffPage {self.index}: '
6665                    f'failed to reshape {result.shape} to {keyframe.shape}'
6666                )
6667
6668        if closed:
6669            # TODO: file should remain open if an exception occurred above
6670            fh.close()
6671        return result
6672
6673    def aszarr(self, **kwargs):
6674        """Return image data as zarr storage."""
6675        return ZarrTiffStore(self, **kwargs)
6676
6677    def asrgb(
6678        self,
6679        uint8=False,
6680        alpha=None,
6681        colormap=None,
6682        dmin=None,
6683        dmax=None,
6684        **kwargs,
6685    ):
6686        """Return image data as RGB(A).
6687
6688        Work in progress.
6689
6690        """
6691        data = self.asarray(**kwargs)
6692        keyframe = self.keyframe  # self or keyframe
6693
6694        if keyframe.photometric == TIFF.PHOTOMETRIC.PALETTE:
6695            colormap = keyframe.colormap
6696            if (
6697                colormap.shape[1] < 2 ** keyframe.bitspersample
6698                or keyframe.dtype.char not in 'BH'
6699            ):
6700                raise ValueError(
6701                    f'TiffPage {self.index}: cannot apply colormap'
6702                )
6703            if uint8:
6704                if colormap.max() > 255:
6705                    colormap >>= 8
6706                colormap = colormap.astype('uint8')
6707            if 'S' in keyframe.axes:
6708                data = data[..., 0] if keyframe.planarconfig == 1 else data[0]
6709            data = apply_colormap(data, colormap)
6710
6711        elif keyframe.photometric == TIFF.PHOTOMETRIC.RGB:
6712            if keyframe.extrasamples:
6713                if alpha is None:
6714                    alpha = TIFF.EXTRASAMPLE
6715                for i, exs in enumerate(keyframe.extrasamples):
6716                    if exs in alpha:
6717                        if keyframe.planarconfig == 1:
6718                            data = data[..., [0, 1, 2, 3 + i]]
6719                        else:
6720                            data = data[:, [0, 1, 2, 3 + i]]
6721                        break
6722            else:
6723                if keyframe.planarconfig == 1:
6724                    data = data[..., :3]
6725                else:
6726                    data = data[:, :3]
6727            # TODO: convert to uint8?
6728
6729        elif keyframe.photometric == TIFF.PHOTOMETRIC.MINISBLACK:
6730            raise NotImplementedError
6731        elif keyframe.photometric == TIFF.PHOTOMETRIC.MINISWHITE:
6732            raise NotImplementedError
6733        elif keyframe.photometric == TIFF.PHOTOMETRIC.SEPARATED:
6734            raise NotImplementedError
6735        else:
6736            raise NotImplementedError
6737        return data
6738
6739    def _gettags(self, codes=None, lock=None):
6740        """Return list of (code, TiffTag)."""
6741        return [
6742            (tag.code, tag)
6743            for tag in self.tags
6744            if codes is None or tag.code in codes
6745        ]
6746
6747    def _nextifd(self):
6748        """Return offset to next IFD from file."""
6749        fh = self.parent.filehandle
6750        tiff = self.parent.tiff
6751        fh.seek(self.offset)
6752        tagno = struct.unpack(tiff.tagnoformat, fh.read(tiff.tagnosize))[0]
6753        fh.seek(self.offset + tiff.tagnosize + tagno * tiff.tagsize)
6754        return struct.unpack(tiff.offsetformat, fh.read(tiff.offsetsize))[0]
6755
6756    def aspage(self):
6757        """Return self."""
6758        return self
6759
6760    @property
6761    def keyframe(self):
6762        """Return keyframe, self."""
6763        return self
6764
6765    @keyframe.setter
6766    def keyframe(self, index):
6767        """Set keyframe, NOP."""
6768        return
6769
6770    @property
6771    def ndim(self):
6772        """Return number of array dimensions."""
6773        return len(self.shape)
6774
6775    @property
6776    def size(self):
6777        """Return number of elements in array."""
6778        return product(self.shape)
6779
6780    @property
6781    def nbytes(self):
6782        """Return number of bytes in array."""
6783        return product(self.shape) * self.dtype.itemsize
6784
6785    @lazyattr
6786    def chunks(self):
6787        """Return shape of tiles or stripes."""
6788        shape = []
6789        if self.tiledepth > 1:
6790            shape.append(self.tiledepth)
6791        if self.is_tiled:
6792            shape.extend((self.tilelength, self.tilewidth))
6793        else:
6794            shape.extend((self.rowsperstrip, self.imagewidth))
6795        if self.planarconfig == 1 and self.samplesperpixel > 1:
6796            shape.append(self.samplesperpixel)
6797        return tuple(shape)
6798
6799    @lazyattr
6800    def chunked(self):
6801        """Return shape of chunked image."""
6802        shape = []
6803        if self.planarconfig == 2 and self.samplesperpixel > 1:
6804            shape.append(self.samplesperpixel)
6805        if self.is_tiled:
6806            if self.imagedepth > 1:
6807                shape.append(
6808                    (self.imagedepth + self.tiledepth - 1) // self.tiledepth
6809                )
6810            shape.append(
6811                (self.imagelength + self.tilelength - 1) // self.tilelength
6812            )
6813            shape.append(
6814                (self.imagewidth + self.tilewidth - 1) // self.tilewidth
6815            )
6816        else:
6817            if self.imagedepth > 1:
6818                shape.append(self.imagedepth)
6819            shape.append(
6820                (self.imagelength + self.rowsperstrip - 1) // self.rowsperstrip
6821            )
6822            shape.append(1)
6823        if self.planarconfig == 1 and self.samplesperpixel > 1:
6824            shape.append(1)
6825        return tuple(shape)
6826
6827    @lazyattr
6828    def hash(self):
6829        """Return checksum to identify pages in same series.
6830
6831        Pages with the same hash can use the same decode function.
6832
6833        """
6834        return hash(
6835            self.shaped
6836            + (
6837                self.parent.byteorder,
6838                self.tilewidth,
6839                self.tilelength,
6840                self.tiledepth,
6841                self.sampleformat,
6842                self.bitspersample,
6843                self.rowsperstrip,
6844                self.fillorder,
6845                self.predictor,
6846                self.extrasamples,
6847                self.photometric,
6848                self.planarconfig,
6849                self.compression,
6850            )
6851        )
6852
6853    @lazyattr
6854    def pages(self):
6855        """Return sequence of sub-pages, SubIFDs."""
6856        if 330 not in self.tags:
6857            return ()
6858        return TiffPages(self, index=self.index)
6859
6860    @lazyattr
6861    def maxworkers(self):
6862        """Return maximum number of threads for decoding segments.
6863
6864        Return 0 to disable multi-threading also for stacking pages.
6865
6866        """
6867        if self.is_contiguous or self.dtype is None:
6868            return 0
6869        if self.compression in (
6870            6,
6871            7,
6872            33003,
6873            33004,
6874            33005,
6875            33007,
6876            34712,
6877            34892,
6878            34933,
6879            34934,
6880            22610,
6881            50001,
6882            50002,
6883        ):
6884            # image codecs
6885            return min(TIFF.MAXWORKERS, len(self.dataoffsets))
6886        bytecount = product(self.chunks) * self.dtype.itemsize
6887        if bytecount < 2048:
6888            # disable multi-threading for small segments
6889            return 0
6890        if self.compression != 1 or self.fillorder != 1 or self.predictor != 1:
6891            if self.compression == 5 and bytecount < 16384:
6892                # disable multi-threading for small LZW compressed segments
6893                return 0
6894        if len(self.dataoffsets) < 4:
6895            return 1
6896        if self.compression != 1 or self.fillorder != 1 or self.predictor != 1:
6897            if imagecodecs is not None:
6898                return min(TIFF.MAXWORKERS, len(self.dataoffsets))
6899        return 2  # optimum for large number of uncompressed tiles
6900
6901    @lazyattr
6902    def is_contiguous(self):
6903        """Return offset and size of contiguous data, else None.
6904
6905        Excludes prediction and fill_order.
6906
6907        """
6908        if self.sampleformat == 5:
6909            return None
6910        if self.compression != 1 or self.bitspersample not in (8, 16, 32, 64):
6911            return None
6912        if 322 in self.tags:  # TileWidth
6913            if (
6914                self.imagewidth != self.tilewidth
6915                or self.imagelength % self.tilelength
6916                or self.tilewidth % 16
6917                or self.tilelength % 16
6918            ):
6919                return None
6920            if (
6921                32997 in self.tags
6922                and 32998 in self.tags  # ImageDepth
6923                and (  # TileDepth
6924                    self.imagelength != self.tilelength
6925                    or self.imagedepth % self.tiledepth
6926                )
6927            ):
6928                return None
6929        offsets = self.dataoffsets
6930        bytecounts = self.databytecounts
6931        if len(offsets) == 1:
6932            return offsets[0], bytecounts[0]
6933        if self.is_stk or self.is_lsm:
6934            return offsets[0], sum(bytecounts)
6935        if all(
6936            bytecounts[i] != 0 and offsets[i] + bytecounts[i] == offsets[i + 1]
6937            for i in range(len(offsets) - 1)
6938        ):
6939            return offsets[0], sum(bytecounts)
6940        return None
6941
6942    @lazyattr
6943    def is_final(self):
6944        """Return if page's image data are stored in final form.
6945
6946        Excludes byte-swapping.
6947
6948        """
6949        return (
6950            self.is_contiguous
6951            and self.fillorder == 1
6952            and self.predictor == 1
6953            and not self.is_subsampled
6954        )
6955
6956    @lazyattr
6957    def is_memmappable(self):
6958        """Return if page's image data in file can be memory-mapped."""
6959        return (
6960            self.parent.filehandle.is_file
6961            and self.is_final
6962            # and (self.bitspersample == 8 or self.parent.isnative)
6963            # aligned?
6964            and self.is_contiguous[0] % self.dtype.itemsize == 0
6965        )
6966
6967    def __str__(self, detail=0, width=79):
6968        """Return string containing information about TiffPage."""
6969        if self.keyframe != self:
6970            return TiffFrame.__str__(self, detail, width)
6971        attr = ''
6972        for name in ('memmappable', 'final', 'contiguous'):
6973            attr = getattr(self, 'is_' + name)
6974            if attr:
6975                attr = name.upper()
6976                break
6977
6978        def tostr(name, skip=1):
6979            obj = getattr(self, name)
6980            try:
6981                value = getattr(obj, 'name')
6982            except AttributeError:
6983                return ''
6984            if obj != skip:
6985                return value
6986            return ''
6987
6988        info = '  '.join(
6989            s.lower()
6990            for s in (
6991                'x'.join(str(i) for i in self.shape),
6992                '{}{}'.format(
6993                    TIFF.SAMPLEFORMAT(self.sampleformat).name,
6994                    self.bitspersample,
6995                ),
6996                ' '.join(
6997                    i
6998                    for i in (
6999                        TIFF.PHOTOMETRIC(self.photometric).name,
7000                        'REDUCED' if self.is_reduced else '',
7001                        'MASK' if self.is_mask else '',
7002                        'TILED' if self.is_tiled else '',
7003                        tostr('compression'),
7004                        tostr('planarconfig'),
7005                        tostr('predictor'),
7006                        tostr('fillorder'),
7007                    )
7008                    + tuple(f.upper() for f in self.flags)
7009                    + (attr,)
7010                    if i
7011                ),
7012            )
7013            if s
7014        )
7015        info = f'TiffPage {self.index} @{self.offset}  {info}'
7016        if detail <= 0:
7017            return info
7018        info = [info, self.tags.__str__(detail + 1, width=width)]
7019        if detail > 1:
7020            for name in ('ndpi',):
7021                name = name + '_tags'
7022                attr = getattr(self, name, False)
7023                if attr:
7024                    info.append(
7025                        '{}\n{}'.format(
7026                            name.upper(),
7027                            pformat(attr, width=width, height=detail * 8),
7028                        )
7029                    )
7030        if detail > 3:
7031            try:
7032                info.append(
7033                    'DATA\n{}'.format(
7034                        pformat(self.asarray(), width=width, height=detail * 8)
7035                    )
7036                )
7037            except Exception:
7038                pass
7039        return '\n\n'.join(info)
7040
7041    @lazyattr
7042    def flags(self):
7043        """Return set of flags."""
7044        return {
7045            name.lower()
7046            for name in sorted(TIFF.FILE_FLAGS)
7047            if getattr(self, 'is_' + name)
7048        }
7049
7050    @lazyattr
7051    def andor_tags(self):
7052        """Return consolidated metadata from Andor tags as dict."""
7053        if not self.is_andor:
7054            return None
7055        result = {'Id': self.tags[4864].value}  # AndorId
7056        for tag in self.tags:  # list(self.tags.values()):
7057            code = tag.code
7058            if not 4864 < code < 5031:
7059                continue
7060            name = tag.name
7061            name = name[5:] if len(name) > 5 else name
7062            result[name] = tag.value
7063            # del self.tags[code]
7064        return result
7065
7066    @lazyattr
7067    def epics_tags(self):
7068        """Return consolidated metadata from EPICS areaDetector tags as dict.
7069
7070        Use epics_datetime() to get a datetime object from the epicsTSSec and
7071        epicsTSNsec tags.
7072
7073        """
7074        if not self.is_epics:
7075            return None
7076        result = {}
7077        for tag in self.tags:  # list(self.tags.values()):
7078            code = tag.code
7079            if not 65000 <= code < 65500:
7080                continue
7081            value = tag.value
7082            if code == 65000:
7083                # not a POSIX timestamp
7084                # https://github.com/bluesky/area-detector-handlers/issues/20
7085                result['timeStamp'] = float(value)
7086            elif code == 65001:
7087                result['uniqueID'] = int(value)
7088            elif code == 65002:
7089                result['epicsTSSec'] = int(value)
7090            elif code == 65003:
7091                result['epicsTSNsec'] = int(value)
7092            else:
7093                key, value = value.split(':', 1)
7094                result[key] = astype(value)
7095            # del self.tags[code]
7096        return result
7097
7098    @lazyattr
7099    def ndpi_tags(self):
7100        """Return consolidated metadata from Hamamatsu NDPI as dict."""
7101        # TODO: parse 65449 ini style comments
7102        if not self.is_ndpi:
7103            return None
7104        tags = self.tags
7105        result = {}
7106        for name in ('Make', 'Model', 'Software'):
7107            result[name] = tags[name].value
7108        for code, name in TIFF.NDPI_TAGS.items():
7109            if code in tags:
7110                result[name] = tags[code].value
7111                # del tags[code]
7112        if 'McuStarts' in result:
7113            mcustarts = result['McuStarts']
7114            if 'McuStartsHighBytes' in result:
7115                high = result['McuStartsHighBytes'].astype('uint64')
7116                high <<= 32
7117                mcustarts = mcustarts.astype('uint64')
7118                mcustarts += high
7119                del result['McuStartsHighBytes']
7120            result['McuStarts'] = mcustarts
7121        return result
7122
7123    @lazyattr
7124    def geotiff_tags(self):
7125        """Return consolidated metadata from GeoTIFF tags as dict."""
7126        if not self.is_geotiff:
7127            return None
7128        tags = self.tags
7129
7130        gkd = tags[34735].value  # GeoKeyDirectoryTag
7131        if gkd[0] != 1:
7132            log_warning('GeoTIFF tags: invalid GeoKeyDirectoryTag')
7133            return {}
7134
7135        result = {
7136            'KeyDirectoryVersion': gkd[0],
7137            'KeyRevision': gkd[1],
7138            'KeyRevisionMinor': gkd[2],
7139            # 'NumberOfKeys': gkd[3],
7140        }
7141        # deltags = ['GeoKeyDirectoryTag']
7142        geokeys = TIFF.GEO_KEYS
7143        geocodes = TIFF.GEO_CODES
7144        for index in range(gkd[3]):
7145            try:
7146                keyid, tagid, count, offset = gkd[
7147                    4 + index * 4 : index * 4 + 8
7148                ]
7149            except Exception as exc:
7150                log_warning(f'GeoTIFF tags: {exc}')
7151                continue
7152            if tagid == 0:
7153                value = offset
7154            else:
7155                try:
7156                    value = tags[tagid].value[offset : offset + count]
7157                except KeyError:
7158                    log_warning(f'GeoTIFF tags: {tagid} not found')
7159                    continue
7160                if tagid == 34737 and count > 1 and value[-1] == '|':
7161                    value = value[:-1]
7162                value = value if count > 1 else value[0]
7163            if keyid in geocodes:
7164                try:
7165                    value = geocodes[keyid](value)
7166                except Exception:
7167                    pass
7168            try:
7169                key = geokeys(keyid).name
7170            except ValueError:
7171                key = keyid
7172            result[key] = value
7173
7174        tag = tags.get(33920)  # IntergraphMatrixTag
7175        if tag is not None:
7176            value = numpy.array(tag.value)
7177            if len(value) == 16:
7178                value = value.reshape((4, 4)).tolist()
7179            result['IntergraphMatrix'] = value
7180
7181        tag = tags.get(33550)  # ModelPixelScaleTag
7182        if tag is not None:
7183            result['ModelPixelScale'] = numpy.array(tag.value).tolist()
7184
7185        tag = tags.get(33922)  # ModelTiepointTag
7186        if tag is not None:
7187            value = numpy.array(tag.value).reshape((-1, 6)).squeeze().tolist()
7188            result['ModelTiepoint'] = value
7189
7190        tag = tags.get(34264)  # ModelTransformationTag
7191        if tag is not None:
7192            value = numpy.array(tag.value).reshape((4, 4)).tolist()
7193            result['ModelTransformation'] = value
7194
7195        # if 33550 in tags and 33922 in tags:
7196        #     sx, sy, sz = tags[33550].value  # ModelPixelScaleTag
7197        #     tiepoints = tags[33922].value  # ModelTiepointTag
7198        #     transforms = []
7199        #     for tp in range(0, len(tiepoints), 6):
7200        #         i, j, k, x, y, z = tiepoints[tp : tp + 6]
7201        #         transforms.append(
7202        #             [
7203        #                 [sx, 0.0, 0.0, x - i * sx],
7204        #                 [0.0, -sy, 0.0, y + j * sy],
7205        #                 [0.0, 0.0, sz, z - k * sz],
7206        #                 [0.0, 0.0, 0.0, 1.0],
7207        #             ]
7208        #         )
7209        #     if len(tiepoints) == 6:
7210        #         transforms = transforms[0]
7211        #     result['ModelTransformation'] = transforms
7212
7213        tag = tags.get(50844)  # RPCCoefficientTag
7214        if tag is not None:
7215            rpcc = tag.value
7216            result['RPCCoefficient'] = {
7217                'ERR_BIAS': rpcc[0],
7218                'ERR_RAND': rpcc[1],
7219                'LINE_OFF': rpcc[2],
7220                'SAMP_OFF': rpcc[3],
7221                'LAT_OFF': rpcc[4],
7222                'LONG_OFF': rpcc[5],
7223                'HEIGHT_OFF': rpcc[6],
7224                'LINE_SCALE': rpcc[7],
7225                'SAMP_SCALE': rpcc[8],
7226                'LAT_SCALE': rpcc[9],
7227                'LONG_SCALE': rpcc[10],
7228                'HEIGHT_SCALE': rpcc[11],
7229                'LINE_NUM_COEFF': rpcc[12:33],
7230                'LINE_DEN_COEFF ': rpcc[33:53],
7231                'SAMP_NUM_COEFF': rpcc[53:73],
7232                'SAMP_DEN_COEFF': rpcc[73:],
7233            }
7234        return result
7235
7236    @property
7237    def is_reduced(self):
7238        """Page is reduced image of another image."""
7239        return self.subfiletype & 0b1
7240
7241    @property
7242    def is_multipage(self):
7243        """Page is part of multi-page image."""
7244        return self.subfiletype & 0b10
7245
7246    @property
7247    def is_mask(self):
7248        """Page is transparency mask for another image."""
7249        return self.subfiletype & 0b100
7250
7251    @property
7252    def is_mrc(self):
7253        """Page is part of Mixed Raster Content."""
7254        return self.subfiletype & 0b1000
7255
7256    @property
7257    def is_tiled(self):
7258        """Page contains tiled image."""
7259        return self.tilewidth > 0  # return 322 in self.tags  # TileWidth
7260
7261    @property
7262    def is_subsampled(self):
7263        """Page contains chroma subsampled image."""
7264        if self.subsampling is not None:
7265            return self.subsampling != (1, 1)
7266        return (
7267            self.compression == 7
7268            and self.planarconfig == 1
7269            and self.photometric in (2, 6)
7270        )
7271
7272    @lazyattr
7273    def is_imagej(self):
7274        """Return ImageJ description if exists, else None."""
7275        for description in (self.description, self.description1):
7276            if not description:
7277                return None
7278            if description[:7] == 'ImageJ=':
7279                return description
7280        return None
7281
7282    @lazyattr
7283    def is_shaped(self):
7284        """Return description containing array shape if exists, else None."""
7285        for description in (self.description, self.description1):
7286            if not description or '"mibi.' in description:
7287                return None
7288            if description[:1] == '{' and '"shape":' in description:
7289                return description
7290            if description[:6] == 'shape=':
7291                return description
7292        return None
7293
7294    @property
7295    def is_mdgel(self):
7296        """Page contains MDFileTag tag."""
7297        return 33445 in self.tags  # MDFileTag
7298
7299    @property
7300    def is_mediacy(self):
7301        """Page contains Media Cybernetics Id tag."""
7302        tag = self.tags.get(50288)  # MC_Id
7303        try:
7304            return tag is not None and tag.value[:7] == b'MC TIFF'
7305        except Exception:
7306            return False
7307
7308    @property
7309    def is_stk(self):
7310        """Page contains UIC2Tag tag."""
7311        return 33629 in self.tags
7312
7313    @property
7314    def is_lsm(self):
7315        """Page contains CZ_LSMINFO tag."""
7316        return 34412 in self.tags
7317
7318    @property
7319    def is_fluoview(self):
7320        """Page contains FluoView MM_STAMP tag."""
7321        return 34362 in self.tags
7322
7323    @property
7324    def is_nih(self):
7325        """Page contains NIHImageHeader tag."""
7326        return 43314 in self.tags
7327
7328    @property
7329    def is_volumetric(self):
7330        """Page contains SGI ImageDepth tag with value > 1."""
7331        return self.imagedepth > 1
7332
7333    @property
7334    def is_vista(self):
7335        """Software tag is 'ISS Vista'."""
7336        return self.software == 'ISS Vista'
7337
7338    @property
7339    def is_metaseries(self):
7340        """Page contains MDS MetaSeries metadata in ImageDescription tag."""
7341        if self.index != 0 or self.software != 'MetaSeries':
7342            return False
7343        d = self.description
7344        return d.startswith('<MetaData>') and d.endswith('</MetaData>')
7345
7346    @property
7347    def is_ome(self):
7348        """Page contains OME-XML in ImageDescription tag."""
7349        if self.index != 0 or not self.description:
7350            return False
7351        return self.description[-4:] == 'OME>'  # and [:13] == '<?xml version'
7352
7353    @property
7354    def is_scn(self):
7355        """Page contains Leica SCN XML in ImageDescription tag."""
7356        if self.index != 0 or not self.description:
7357            return False
7358        return self.description[-6:] == '</scn>'
7359
7360    @property
7361    def is_micromanager(self):
7362        """Page contains MicroManagerMetadata tag."""
7363        return 51123 in self.tags
7364
7365    @property
7366    def is_andor(self):
7367        """Page contains Andor Technology tags 4864-5030."""
7368        return 4864 in self.tags
7369
7370    @property
7371    def is_pilatus(self):
7372        """Page contains Pilatus tags."""
7373        return self.software[:8] == 'TVX TIFF' and self.description[:2] == '# '
7374
7375    @property
7376    def is_epics(self):
7377        """Page contains EPICS areaDetector tags."""
7378        return (
7379            self.description == 'EPICS areaDetector'
7380            or self.software == 'EPICS areaDetector'
7381        )
7382
7383    @property
7384    def is_tvips(self):
7385        """Page contains TVIPS metadata."""
7386        return 37706 in self.tags
7387
7388    @property
7389    def is_fei(self):
7390        """Page contains FEI_SFEG or FEI_HELIOS tags."""
7391        return 34680 in self.tags or 34682 in self.tags
7392
7393    @property
7394    def is_sem(self):
7395        """Page contains CZ_SEM tag."""
7396        return 34118 in self.tags
7397
7398    @property
7399    def is_svs(self):
7400        """Page contains Aperio metadata."""
7401        return self.description[:7] == 'Aperio '
7402
7403    @property
7404    def is_bif(self):
7405        """Page contains Ventana metadata."""
7406        try:
7407            return (
7408                self.description == 'Label Image'
7409                or ('Ventana' in self.software)
7410            ) and (
7411                700 in self.tags and b'<iScan' in self.tags[700].value[:4096]
7412            )
7413        except Exception:
7414            return False
7415
7416    @property
7417    def is_scanimage(self):
7418        """Page contains ScanImage metadata."""
7419        return (
7420            self.software[:3] == 'SI.'
7421            or self.description[:6] == 'state.'
7422            or 'scanimage.SI' in self.description[-256:]
7423        )
7424
7425    @property
7426    def is_qpi(self):
7427        """Page contains PerkinElmer tissue images metadata."""
7428        # The ImageDescription tag contains XML with a top-level
7429        # <PerkinElmer-QPI-ImageDescription> element
7430        return self.software[:15] == 'PerkinElmer-QPI'
7431
7432    @property
7433    def is_geotiff(self):
7434        """Page contains GeoTIFF metadata."""
7435        return 34735 in self.tags  # GeoKeyDirectoryTag
7436
7437    @property
7438    def is_tiffep(self):
7439        """Page contains TIFF/EP metadata."""
7440        return 37398 in self.tags  # TIFF/EPStandardID
7441
7442    @property
7443    def is_sis(self):
7444        """Page contains Olympus SIS metadata."""
7445        return 33560 in self.tags or 33471 in self.tags
7446
7447    @property
7448    def is_ndpi(self):
7449        """Page contains NDPI metadata."""
7450        return 65420 in self.tags and 271 in self.tags
7451
7452    @property
7453    def is_philips(self):
7454        """Page contains Philips DP metadata."""
7455        return (
7456            self.software[:10] == 'Philips DP'
7457            and self.description[-13:] == '</DataObject>'
7458        )
7459
7460    @property
7461    def is_eer(self):
7462        """Page contains EER metadata."""
7463        return (
7464            self.parent.is_bigtiff
7465            and self.compression in (65000, 65001)
7466            and 65001 in self.tags
7467        )
7468
7469
7470class TiffFrame:
7471    """Lightweight TIFF image file directory (IFD).
7472
7473    Only a limited number of tag values are read from file.
7474    Other tag values are assumed to be identical with a specified TiffPage
7475    instance, the keyframe.
7476
7477    TiffFrame is intended to reduce resource usage and speed up reading image
7478    data from file, not for introspection of metadata.
7479
7480    """
7481
7482    __slots__ = (
7483        'index',
7484        'parent',
7485        'offset',
7486        'dataoffsets',
7487        'databytecounts',
7488        'subifds',
7489        'jpegtables',
7490        '_keyframe',
7491    )
7492
7493    is_mdgel = False
7494    pages = ()
7495    # tags = {}
7496
7497    def __init__(
7498        self,
7499        parent,
7500        index,
7501        offset=None,
7502        keyframe=None,
7503        offsets=None,
7504        bytecounts=None,
7505    ):
7506        """Initialize TiffFrame from file or values.
7507
7508        The file handle position must be at the offset to a valid IFD.
7509
7510        """
7511        self._keyframe = None
7512        self.parent = parent
7513        self.index = index
7514        self.offset = offset
7515        self.subifds = None
7516        self.jpegtables = None
7517        self.dataoffsets = ()
7518        self.databytecounts = ()
7519
7520        if offsets is not None:
7521            # initialize "virtual frame" from offsets and bytecounts
7522            self.dataoffsets = offsets
7523            self.databytecounts = bytecounts
7524            self._keyframe = keyframe
7525            return
7526
7527        if offset is None:
7528            self.offset = self.parent.filehandle.tell()
7529        else:
7530            self.parent.filehandle.seek(offset)
7531
7532        if keyframe is None:
7533            tags = {273, 279, 324, 325, 330, 347}
7534        elif keyframe.is_contiguous:
7535            # use databytecounts from keyframe
7536            tags = {256, 273, 324, 330}
7537            self.databytecounts = keyframe.databytecounts
7538        else:
7539            tags = {256, 273, 279, 324, 325, 330, 347}
7540
7541        for code, tag in self._gettags(tags):
7542            if code == 273 or code == 324:
7543                self.dataoffsets = tag.value
7544            elif code == 279 or code == 325:
7545                self.databytecounts = tag.value
7546            elif code == 330:
7547                self.subifds = tag.value
7548            elif code == 347:
7549                self.jpegtables = tag.value
7550            elif code == 256 and keyframe.imagewidth != tag.value:
7551                raise RuntimeError(
7552                    f'TiffFrame {self.index} incompatible keyframe'
7553                )
7554
7555        if not self.dataoffsets:
7556            log_warning(f'TiffFrame {self.index}: missing required tags')
7557
7558        self.keyframe = keyframe
7559
7560    def _gettags(self, codes=None, lock=None):
7561        """Return list of (code, TiffTag) from file."""
7562        fh = self.parent.filehandle
7563        tiff = self.parent.tiff
7564        unpack = struct.unpack
7565        lock = NullContext() if lock is None else lock
7566        tags = []
7567
7568        with lock:
7569            fh.seek(self.offset)
7570            try:
7571                tagno = unpack(tiff.tagnoformat, fh.read(tiff.tagnosize))[0]
7572                if tagno > 4096:
7573                    raise ValueError(f'suspicious number of tags {tagno}')
7574            except Exception as exc:
7575                raise TiffFileError(
7576                    f'TiffFrame {self.index}: '
7577                    f'corrupted tag list at offset {self.offset}'
7578                ) from exc
7579
7580            tagoffset = self.offset + tiff.tagnosize  # fh.tell()
7581            tagsize = tiff.tagsize
7582            tagindex = -tagsize
7583            codeformat = tiff.tagformat1[:2]
7584            tagbytes = fh.read(tagsize * tagno)
7585
7586            for _ in range(tagno):
7587                tagindex += tagsize
7588                code = unpack(codeformat, tagbytes[tagindex : tagindex + 2])[0]
7589                if codes and code not in codes:
7590                    continue
7591                try:
7592                    tag = TiffTag.fromfile(
7593                        self.parent,
7594                        tagoffset + tagindex,
7595                        tagbytes[tagindex : tagindex + tagsize],
7596                    )
7597                except TiffFileError as exc:
7598                    log_warning(
7599                        f'TiffFrame {self.index}: '
7600                        f'{exc.__class__.__name__}: {exc}'
7601                    )
7602                    continue
7603                tags.append((code, tag))
7604
7605        return tags
7606
7607    def _nextifd(self):
7608        """Return offset to next IFD from file."""
7609        return TiffPage._nextifd(self)
7610
7611    def aspage(self):
7612        """Return TiffPage from file."""
7613        if self.offset is None:
7614            raise ValueError(
7615                f'TiffFrame {self.index}: cannot return virtual frame as page'
7616            )
7617        self.parent.filehandle.seek(self.offset)
7618        return TiffPage(self.parent, index=self.index)
7619
7620    def asarray(self, *args, **kwargs):
7621        """Read image data from file and return as numpy array."""
7622        return TiffPage.asarray(self, *args, **kwargs)
7623
7624    def aszarr(self, **kwargs):
7625        """Return image data as zarr storage."""
7626        return TiffPage.aszarr(self, **kwargs)
7627
7628    def asrgb(self, *args, **kwargs):
7629        """Read image data from file and return RGB image as numpy array."""
7630        return TiffPage.asrgb(self, *args, **kwargs)
7631
7632    def segments(self, *args, **kwargs):
7633        """Return iterator over decoded segments in TiffFrame."""
7634        return TiffPage.segments(self, *args, **kwargs)
7635
7636    @property
7637    def keyframe(self):
7638        """Return keyframe."""
7639        return self._keyframe
7640
7641    @keyframe.setter
7642    def keyframe(self, keyframe):
7643        """Set keyframe."""
7644        if self._keyframe == keyframe:
7645            return
7646        if self._keyframe is not None:
7647            raise RuntimeError(
7648                f'TiffFrame {self.index}: cannot reset keyframe'
7649            )
7650        if len(self.dataoffsets) != len(keyframe.dataoffsets):
7651            raise RuntimeError(
7652                f'TiffFrame {self.index}: incompatible keyframe'
7653            )
7654        if keyframe.is_contiguous:
7655            self.databytecounts = keyframe.databytecounts
7656        self._keyframe = keyframe
7657
7658    @property
7659    def is_contiguous(self):
7660        """Return offset and size of contiguous data, else None."""
7661        # if self._keyframe is None:
7662        #     raise RuntimeError(f'TiffFrame {self.index}: keyframe not set')
7663        if self._keyframe.is_contiguous:
7664            return self.dataoffsets[0], self._keyframe.is_contiguous[1]
7665        return None
7666
7667    @property
7668    def is_memmappable(self):
7669        """Return if page's image data in file can be memory-mapped."""
7670        # if self._keyframe is None:
7671        #     raise RuntimeError(f'TiffFrame {self.index}: keyframe not set')
7672        return self._keyframe.is_memmappable
7673
7674    @property
7675    def hash(self):
7676        """Return checksum to identify pages in same series."""
7677        # if self._keyframe is None:
7678        #     raise RuntimeError(f'TiffFrame {self.index}: keyframe not set')
7679        return self._keyframe.hash
7680
7681    def __getattr__(self, name):
7682        """Return attribute from keyframe."""
7683        if name in TIFF.FRAME_ATTRS:
7684            return getattr(self._keyframe, name)
7685        # this error could be raised because an AttributeError was
7686        # raised inside a @property function
7687        raise AttributeError(
7688            f'{self.__class__.__name__!r} object has no attribute {name!r}'
7689        )
7690
7691    def __str__(self, detail=0, width=79):
7692        """Return string containing information about TiffFrame."""
7693        if self._keyframe is None:
7694            info = ''
7695            kf = None
7696        else:
7697            info = '  '.join(
7698                s
7699                for s in (
7700                    'x'.join(str(i) for i in self.shape),
7701                    str(self.dtype),
7702                )
7703            )
7704            kf = TiffPage.__str__(self._keyframe, width=width - 11)
7705        if detail > 3:
7706            of = pformat(self.dataoffsets, width=width - 9, height=detail - 3)
7707            bc = pformat(
7708                self.databytecounts, width=width - 13, height=detail - 3
7709            )
7710            info = f'\n Keyframe {kf}\n Offsets {of}\n Bytecounts {bc}'
7711        return f'TiffFrame {self.index} @{self.offset}  {info}'
7712
7713
7714class TiffTag:
7715    """TIFF tag structure.
7716
7717    Attributes
7718    ----------
7719    name : string
7720        Name of tag, TIFF.TAGS[code].
7721    code : int
7722        Decimal code of tag.
7723    dtype : int
7724        Datatype of tag data. One of TIFF.DATATYPES.
7725    count : int
7726        Number of values.
7727    value : various types
7728        Tag data as Python object.
7729    valueoffset : int
7730        Location of value in file.
7731    offset : int
7732        Location of tag structure in file.
7733    parent : TiffFile or TiffWriter
7734        Reference to parent TIFF file.
7735
7736    All attributes are read-only.
7737
7738    """
7739
7740    __slots__ = (
7741        'parent',
7742        'offset',
7743        'code',
7744        'dtype',
7745        'count',
7746        'value',
7747        'valueoffset',
7748    )
7749
7750    def __init__(self, parent, offset, code, dtype, count, value, valueoffset):
7751        """Initialize TiffTag instance from values."""
7752        self.parent = parent
7753        self.offset = offset
7754        self.code = code
7755        self.dtype = TIFF.DATATYPES(dtype)
7756        self.count = count
7757        self.value = value
7758        self.valueoffset = valueoffset
7759
7760    @classmethod
7761    def fromfile(cls, parent, offset=None, header=None):
7762        """Return TiffTag instance read from file."""
7763        fh = parent.filehandle
7764        tiff = parent.tiff
7765        unpack = struct.unpack
7766
7767        if header is None:
7768            if offset is None:
7769                offset = fh.tell()
7770            else:
7771                fh.seek(offset)
7772            header = fh.read(tiff.tagsize)
7773        elif offset is None:
7774            offset = fh.tell()
7775
7776        valueoffset = offset + tiff.tagsize - tiff.tagoffsetthreshold
7777        code, dtype = unpack(tiff.tagformat1, header[:4])
7778        count, value = unpack(tiff.tagformat2, header[4:])
7779
7780        try:
7781            dataformat = TIFF.DATA_FORMATS[dtype]
7782        except KeyError:
7783            raise TiffFileError(f'unknown tag data type {dtype!r}')
7784
7785        fmt = '{}{}{}'.format(
7786            tiff.byteorder, count * int(dataformat[0]), dataformat[1]
7787        )
7788        size = struct.calcsize(fmt)
7789        if size > tiff.tagoffsetthreshold or code in TIFF.TAG_READERS:
7790            valueoffset = unpack(tiff.offsetformat, value)[0]
7791            if valueoffset < 8 or valueoffset > fh.size - size:
7792                raise TiffFileError('invalid tag value offset')
7793            # if valueoffset % 2:
7794            #     log_warning('TiffTag: value does not begin on word boundary')
7795            fh.seek(valueoffset)
7796            if code in TIFF.TAG_READERS:
7797                readfunc = TIFF.TAG_READERS[code]
7798                value = readfunc(
7799                    fh, tiff.byteorder, dtype, count, tiff.offsetsize
7800                )
7801            elif dtype == 7 or (count > 1 and dtype == 1):
7802                value = read_bytes(
7803                    fh, tiff.byteorder, dtype, count, tiff.offsetsize
7804                )
7805            # elif code in TIFF.TAGS or dtype == 2:
7806            else:
7807                value = unpack(fmt, fh.read(size))
7808            # else:
7809            #     value = read_numpy(
7810            #         fh, tiff.byteorder, dtype, count, tiff.offsetsize
7811            #     )
7812        elif dtype in (1, 7):
7813            # BYTES, UNDEFINED
7814            value = value[:size]
7815        elif (
7816            tiff.version == 42
7817            and tiff.offsetsize == 8
7818            and count == 1
7819            and dtype in (4, 13)
7820            and value[4:] != b'\x00\x00\x00\x00'
7821        ):
7822            # NDPI LONG or IFD
7823            # dtype = 16 if dtype == 4 else 18
7824            # dataformat = '1Q'
7825            value = unpack('<Q', value)
7826        else:
7827            value = unpack(fmt, value[:size])
7828
7829        process = (
7830            code not in TIFF.TAG_READERS
7831            and code not in TIFF.TAG_TUPLE
7832            and dtype != 7  # UNDEFINED
7833        )
7834        if process and dtype == 2 and isinstance(value[0], bytes):
7835            # TIFF ASCII fields can contain multiple strings,
7836            #   each terminated with a NUL
7837            value = value[0]
7838            try:
7839                value = bytes2str(stripnull(value, first=False).strip())
7840            except UnicodeDecodeError:
7841                log_warning(f'TiffTag {code}: coercing invalid ASCII to bytes')
7842                # dtype = 1
7843                # dataformat = '1B'
7844        else:
7845            if code in TIFF.TAG_ENUM:
7846                t = TIFF.TAG_ENUM[code]
7847                try:
7848                    value = tuple(t(v) for v in value)
7849                except ValueError as exc:
7850                    if code not in (259, 317):  # ignore compression/predictor
7851                        log_warning(f'TiffTag {code}: {exc}')
7852            if process:
7853                if len(value) == 1:
7854                    value = value[0]
7855        return cls(parent, offset, code, dtype, count, value, valueoffset)
7856
7857    @property
7858    def name(self):
7859        """Return name of tag from TIFF.TAGS registry."""
7860        return TIFF.TAGS.get(self.code, str(self.code))
7861
7862    @property
7863    def dataformat(self):
7864        """Return data type as Python struct format."""
7865        return TIFF.DATA_FORMATS[self.dtype]
7866
7867    @property
7868    def valuebytecount(self):
7869        """Return size of value in file."""
7870        return self.count * struct.calcsize(TIFF.DATA_FORMATS[self.dtype])
7871
7872    def _astuple(self):
7873        """Return tag code, dtype, count, and encoded value.
7874
7875        The encoded value is read from file if necessary.
7876
7877        """
7878        # TODO: make this method public
7879        if isinstance(self.value, bytes):
7880            value = self.value
7881        else:
7882            dataformat = TIFF.DATA_FORMATS[self.dtype]
7883            count = self.count * int(dataformat[0])
7884            fmt = '{}{}{}'.format(
7885                self.parent.tiff.byteorder, count, dataformat[1]
7886            )
7887            try:
7888                if count == 1:
7889                    value = struct.pack(fmt, self.value)
7890                else:
7891                    value = struct.pack(fmt, *self.value)
7892            except Exception:
7893                tiff = self.parent.tiff
7894                if tiff.version == 42 and tiff.offsetsize == 8:
7895                    raise NotImplementedError('cannot read NDPI > 4 GB files')
7896                fh = self.parent.filehandle
7897                pos = fh.tell()
7898                fh.seek(self.valueoffset)
7899                value = fh.read(struct.calcsize(fmt))
7900                fh.seek(pos)
7901        return self.code, self.dtype.value, self.count, value
7902
7903    def overwrite(self, value, _arg=None, dtype=None, erase=True):
7904        """Write new tag value to file and return new TiffTag instance.
7905
7906        The value must be compatible with the struct.pack formats in
7907        TIFF.DATA_FORMATS.
7908
7909        The new packed value is appended to the file if it is longer than the
7910        old value. The old value is zeroed. The file position is left where it
7911        was.
7912
7913        """
7914        if self.offset < 8 or self.valueoffset < 8:
7915            raise ValueError('cannot rewrite tag at offset < 8')
7916
7917        if hasattr(value, 'filehandle'):
7918            value = _arg
7919            warnings.warn(
7920                "TiffTag.overwrite: passing a TiffFile instance is deprecated "
7921                "and no longer required since 2021.7.x.",
7922                DeprecationWarning,
7923                stacklevel=2,
7924            )
7925
7926        fh = self.parent.filehandle
7927        tiff = self.parent.tiff
7928
7929        if tiff.version == 42 and tiff.offsetsize == 8:
7930            # TODO: support patching NDPI > 4 GB files
7931            raise NotImplementedError('cannot patch NDPI > 4 GB files')
7932
7933        if value is None:
7934            value = b''
7935        if dtype is None:
7936            dtype = self.dtype
7937
7938        try:
7939            dataformat = TIFF.DATA_FORMATS[dtype]
7940        except KeyError as exc:
7941            try:
7942                dataformat = dtype[-1:]
7943                if dataformat[0] in '<>':
7944                    dataformat = dataformat[1:]
7945                dtype = TIFF.DATA_DTYPES[dataformat]
7946            except (KeyError, TypeError):
7947                raise ValueError(f'unknown dtype {dtype}') from exc
7948
7949        if dtype == 2:
7950            # strings
7951            if isinstance(value, str):
7952                # enforce 7-bit ASCII on Unicode strings
7953                try:
7954                    value = value.encode('ascii')
7955                except UnicodeEncodeError as exc:
7956                    raise ValueError(
7957                        'TIFF strings must be 7-bit ASCII'
7958                    ) from exc
7959            elif not isinstance(value, bytes):
7960                raise ValueError('TIFF strings must be 7-bit ASCII')
7961            if len(value) == 0 or value[-1] != b'\x00':
7962                value += b'\x00'
7963            count = len(value)
7964            value = (value,)
7965        elif isinstance(value, bytes):
7966            # pre-packed binary data
7967            dtsize = struct.calcsize(dataformat)
7968            if len(value) % dtsize:
7969                raise ValueError('invalid packed binary data')
7970            count = len(value) // dtsize
7971            value = (value,)
7972        else:
7973            try:
7974                count = len(value)
7975            except TypeError:
7976                value = (value,)
7977                count = 1
7978
7979        if dtype in (5, 10):
7980            if count < 2 or count % 2:
7981                raise ValueError('invalid RATIONAL value')
7982            count //= 2  # rational
7983
7984        packedvalue = struct.pack(
7985            '{}{}{}'.format(
7986                tiff.byteorder, count * int(dataformat[0]), dataformat[1]
7987            ),
7988            *value,
7989        )
7990        newsize = len(packedvalue)
7991        oldsize = self.count * struct.calcsize(TIFF.DATA_FORMATS[self.dtype])
7992        valueoffset = self.valueoffset
7993
7994        pos = fh.tell()
7995        try:
7996            if dtype != self.dtype:
7997                # rewrite data type
7998                fh.seek(self.offset + 2)
7999                fh.write(struct.pack(tiff.byteorder + 'H', dtype))
8000
8001            if oldsize <= tiff.tagoffsetthreshold:
8002                if newsize <= tiff.tagoffsetthreshold:
8003                    # inline -> inline: overwrite
8004                    fh.seek(self.offset + 4)
8005                    fh.write(struct.pack(tiff.tagformat2, count, packedvalue))
8006                else:
8007                    # inline -> separate: append to file
8008                    fh.seek(0, os.SEEK_END)
8009                    valueoffset = fh.tell()
8010                    if valueoffset % 2:
8011                        # value offset must begin on a word boundary
8012                        fh.write(b'\x00')
8013                        valueoffset += 1
8014                    fh.write(packedvalue)
8015                    fh.seek(self.offset + 4)
8016                    fh.write(
8017                        struct.pack(
8018                            tiff.tagformat2,
8019                            count,
8020                            struct.pack(tiff.offsetformat, valueoffset),
8021                        )
8022                    )
8023            elif newsize <= tiff.tagoffsetthreshold:
8024                # separate -> inline: erase old value
8025                valueoffset = self.offset + 4 + tiff.offsetsize
8026                fh.seek(self.offset + 4)
8027                fh.write(struct.pack(tiff.tagformat2, count, packedvalue))
8028                if erase:
8029                    fh.seek(self.valueoffset)
8030                    fh.write(b'\x00' * oldsize)
8031            elif newsize <= oldsize or self.valueoffset + oldsize == fh.size:
8032                # separate -> separate smaller: overwrite, erase remaining
8033                fh.seek(self.offset + 4)
8034                fh.write(struct.pack(tiff.offsetformat, count))
8035                fh.seek(self.valueoffset)
8036                fh.write(packedvalue)
8037                if erase and oldsize - newsize > 0:
8038                    fh.write(b'\x00' * (oldsize - newsize))
8039            else:
8040                # separate -> separate larger: erase old value, append to file
8041                if erase:
8042                    fh.seek(self.valueoffset)
8043                    fh.write(b'\x00' * oldsize)
8044                fh.seek(0, os.SEEK_END)
8045                valueoffset = fh.tell()
8046                if valueoffset % 2:
8047                    # value offset must begin on a word boundary
8048                    fh.write(b'\x00')
8049                    valueoffset += 1
8050                fh.write(packedvalue)
8051                fh.seek(self.offset + 4)
8052                fh.write(
8053                    struct.pack(
8054                        tiff.tagformat2,
8055                        count,
8056                        struct.pack(tiff.offsetformat, valueoffset),
8057                    )
8058                )
8059        finally:
8060            fh.seek(pos)  # must restore file position
8061
8062        return TiffTag(
8063            self.parent,
8064            self.offset,
8065            self.code,
8066            dtype,
8067            count,
8068            value,
8069            valueoffset,
8070        )
8071
8072    def _fix_lsm_bitspersample(self):
8073        """Correct LSM bitspersample tag.
8074
8075        Old LSM writers may use a separate region for two 16-bit values,
8076        although they fit into the tag value element of the tag.
8077
8078        """
8079        if self.code != 258 or self.count != 2:
8080            return
8081        # TODO: test this case; need example file
8082        log_warning(f'TiffTag {self.code}: correcting LSM bitspersample tag')
8083        value = struct.pack('<HH', *self.value)
8084        self.valueoffset = struct.unpack('<I', value)[0]
8085        self.parent.filehandle.seek(self.valueoffset)
8086        self.value = struct.unpack('<HH', self.parent.filehandle.read(4))
8087
8088    def __str__(self, detail=0, width=79):
8089        """Return string containing information about TiffTag."""
8090        height = 1 if detail <= 0 else 8 * detail
8091        dtype = self.dtype.name
8092        if self.count > 1:
8093            dtype += f'[{self.count}]'
8094        name = '|'.join(TIFF.TAGS.getall(self.code, ()))
8095        if name:
8096            name = f'{self.code} {name} @{self.offset}'
8097        else:
8098            name = f'{self.code} @{self.offset}'
8099        line = f'TiffTag {name} {dtype} @{self.valueoffset} '
8100        line = line[:width]
8101        try:
8102            if self.count == 1:
8103                value = enumstr(self.value)
8104            else:
8105                value = pformat(tuple(enumstr(v) for v in self.value))
8106        except Exception:
8107            value = pformat(self.value, width=width, height=height)
8108        if detail <= 0:
8109            line += value
8110            line = line[:width]
8111        else:
8112            line += '\n' + value
8113        return line
8114
8115
8116class TiffTags:
8117    """Multidict like interface to TiffTag instances in TiffPage.
8118
8119    Differences to a regular dict:
8120
8121    * values are instances of TiffTag.
8122    * keys are TiffTag.code (int).
8123    * multiple values can be stored per key.
8124    * can be indexed with TiffTag.name (str), although slower than by key.
8125    * iter() returns values instead of keys.
8126    * values() and items() contain all values sorted by offset stored in file.
8127    * len() returns the number of all values.
8128    * get() takes an optional index argument.
8129    * some functions are not implemented, e.g. update, setdefault, pop.
8130
8131    """
8132
8133    __slots__ = ('_dict', '_list')
8134
8135    def __init__(self):
8136        """Initialize empty instance."""
8137        self._dict = {}
8138        self._list = [self._dict]
8139
8140    def add(self, tag):
8141        """Add a tag."""
8142        code = tag.code
8143        for d in self._list:
8144            if code not in d:
8145                d[code] = tag
8146                break
8147        else:
8148            self._list.append({code: tag})
8149
8150    def keys(self):
8151        """Return new view of all codes."""
8152        return self._dict.keys()
8153
8154    def values(self):
8155        """Return all tags in order they are stored in file."""
8156        tags = (t for d in self._list for t in d.values())
8157        return sorted(tags, key=lambda t: t.offset)
8158
8159    def items(self):
8160        """Return all (code, tag) pairs in order tags are stored in file."""
8161        items = (i for d in self._list for i in d.items())
8162        return sorted(items, key=lambda i: i[1].offset)
8163
8164    def get(self, key, default=None, index=None):
8165        """Return tag of code or name if exists, else default."""
8166        if index is None:
8167            if key in self._dict:
8168                return self._dict[key]
8169            if not isinstance(key, str):
8170                return default
8171            index = 0
8172        try:
8173            tags = self._list[index]
8174        except IndexError:
8175            return default
8176        if key in tags:
8177            return tags[key]
8178        if not isinstance(key, str):
8179            return default
8180        for tag in tags.values():
8181            if tag.name == key:
8182                return tag
8183        return default
8184
8185    def getall(self, key, default=None):
8186        """Return list of all tags of code or name if exists, else default."""
8187        result = []
8188        for tags in self._list:
8189            if key in tags:
8190                result.append(tags[key])
8191            else:
8192                break
8193        if result:
8194            return result
8195        if not isinstance(key, str):
8196            return default
8197        for tags in self._list:
8198            for tag in tags.values():
8199                if tag.name == key:
8200                    result.append(tag)
8201                    break
8202            if not result:
8203                break
8204        return result if result else default
8205
8206    def __getitem__(self, key):
8207        """Return first tag of code or name. Raise KeyError if not found."""
8208        if key in self._dict:
8209            return self._dict[key]
8210        if not isinstance(key, str):
8211            raise KeyError(key)
8212        for tag in self._dict.values():
8213            if tag.name == key:
8214                return tag
8215        raise KeyError(key)
8216
8217    def __setitem__(self, code, tag):
8218        """Add a tag."""
8219        self.add(tag)
8220
8221    def __delitem__(self, key):
8222        """Delete all tags of code or name."""
8223        found = False
8224        for tags in self._list:
8225            if key in tags:
8226                found = True
8227                del tags[key]
8228            else:
8229                break
8230        if found:
8231            return None
8232        if not isinstance(key, str):
8233            raise KeyError(key)
8234        for tags in self._list:
8235            for tag in tags.values():
8236                if tag.name == key:
8237                    del tags[tag.code]
8238                    found = True
8239                    break
8240            else:
8241                break
8242        if not found:
8243            raise KeyError(key)
8244        return None
8245
8246    def __contains__(self, item):
8247        """Return if tag is in map."""
8248        if item in self._dict:
8249            return True
8250        if not isinstance(item, str):
8251            return False
8252        for tag in self._dict.values():
8253            if tag.name == item:
8254                return True
8255        return False
8256
8257    def __iter__(self):
8258        """Return iterator over all tags."""
8259        return iter(self.values())
8260
8261    def __len__(self):
8262        """Return number of tags."""
8263        size = 0
8264        for d in self._list:
8265            size += len(d)
8266        return size
8267
8268    def __str__(self, detail=0, width=79):
8269        """Return string with information about TiffTags."""
8270        info = []
8271        tlines = []
8272        vlines = []
8273        for tag in self:
8274            value = tag.__str__(width=width + 1)
8275            tlines.append(value[:width].strip())
8276            if detail > 0 and len(value) > width:
8277                if detail < 2 and tag.code in (273, 279, 324, 325):
8278                    value = pformat(tag.value, width=width, height=detail * 4)
8279                else:
8280                    value = pformat(tag.value, width=width, height=detail * 12)
8281                vlines.append(f'{tag.name}\n{value}')
8282        info.append('\n'.join(tlines))
8283        if detail > 0 and vlines:
8284            info.append('\n')
8285            info.append('\n\n'.join(vlines))
8286        return '\n'.join(info)
8287
8288
8289class TiffTagRegistry:
8290    """Registry of TIFF tag codes and names.
8291
8292    The registry allows to look up tag codes and names by indexing with names
8293    and codes respectively.
8294    One tag code may be registered with several names, e.g. 34853 is used for
8295    GPSTag or OlympusSIS2.
8296    Different tag codes may be registered with the same name, e.g. 37387 and
8297    41483 are both named FlashEnergy.
8298
8299    """
8300
8301    __slots__ = ('_dict', '_list')
8302
8303    def __init__(self, arg):
8304        self._dict = {}
8305        self._list = [self._dict]
8306        self.update(arg)
8307
8308    def update(self, arg):
8309        """Add codes and names from sequence or dict to registry."""
8310        if isinstance(arg, TiffTagRegistry):
8311            self._list.extend(arg._list)
8312            return
8313        if isinstance(arg, dict):
8314            arg = arg.items()
8315        for code, name in arg:
8316            self.add(code, name)
8317
8318    def add(self, code, name):
8319        """Add code and name to registry."""
8320        for d in self._list:
8321            if code in d and d[code] == name:
8322                break
8323            if code not in d and name not in d:
8324                d[code] = name
8325                d[name] = code
8326                break
8327        else:
8328            self._list.append({code: name, name: code})
8329
8330    def items(self):
8331        """Return all registry items as (code, name)."""
8332        items = (
8333            i for d in self._list for i in d.items() if isinstance(i[0], int)
8334        )
8335        return sorted(items, key=lambda i: i[0])
8336
8337    def get(self, key, default=None):
8338        """Return first code/name if exists, else default."""
8339        for d in self._list:
8340            if key in d:
8341                return d[key]
8342        return default
8343
8344    def getall(self, key, default=None):
8345        """Return list of all codes/names if exists, else default."""
8346        result = [d[key] for d in self._list if key in d]
8347        return result if result else default
8348
8349    def __getitem__(self, key):
8350        """Return first code/name. Raise KeyError if not found."""
8351        for d in self._list:
8352            if key in d:
8353                return d[key]
8354        raise KeyError(key)
8355
8356    def __delitem__(self, key):
8357        """Delete all tags of code or name."""
8358        found = False
8359        for d in self._list:
8360            if key in d:
8361                found = True
8362                value = d[key]
8363                del d[key]
8364                del d[value]
8365        if not found:
8366            raise KeyError(key)
8367
8368    def __contains__(self, item):
8369        """Return if code or name is in registry."""
8370        for d in self._list:
8371            if item in d:
8372                return True
8373        return False
8374
8375    def __iter__(self):
8376        """Return iterator over all items in registry."""
8377        return iter(self.items())
8378
8379    def __len__(self):
8380        """Return number of registered tags."""
8381        size = 0
8382        for d in self._list:
8383            size += len(d)
8384        return size // 2
8385
8386    def __str__(self):
8387        """Return string with information about TiffTags."""
8388        return 'TiffTagRegistry(((\n  {}\n))'.format(
8389            ',\n  '.join(f'({code}, {name!r})' for code, name in self.items())
8390        )
8391
8392
8393class TiffPageSeries:
8394    """Series of TIFF pages with compatible shape and data type.
8395
8396    Attributes
8397    ----------
8398    pages : list of TiffPage
8399        Sequence of TiffPages in series.
8400    dtype : numpy.dtype
8401        Data type (native byte order) of the image array in series.
8402    shape : tuple
8403        Dimensions of the image array in series.
8404    axes : str
8405        Labels of axes in shape. See TIFF.AXES_LABELS.
8406    offset : int or None
8407        Position of image data in file if memory-mappable, else None.
8408    levels : list of TiffPageSeries
8409        Pyramid levels. levels[0] is 'self'.
8410
8411    """
8412
8413    def __init__(
8414        self,
8415        pages,
8416        shape=None,
8417        dtype=None,
8418        axes=None,
8419        parent=None,
8420        name=None,
8421        transform=None,
8422        kind=None,
8423        truncated=False,
8424        multifile=False,
8425        squeeze=True,
8426    ):
8427        """Initialize instance."""
8428        self.index = 0
8429        self._pages = pages  # might contain only first of contiguous pages
8430        self.levels = [self]
8431        if shape is None:
8432            shape = pages[0].shape
8433        if axes is None:
8434            axes = pages[0].axes
8435        if dtype is None:
8436            dtype = pages[0].dtype
8437
8438        self.set_shape_axes(shape, axes, squeeze)
8439
8440        self.dtype = numpy.dtype(dtype)
8441        self.kind = kind if kind else ''
8442        self.name = name if name else ''
8443        self.transform = transform
8444        self.keyframe = next(p.keyframe for p in pages if p is not None)
8445        self.is_multifile = bool(multifile)
8446
8447        if parent:
8448            self.parent = parent
8449        elif pages:
8450            self.parent = self.keyframe.parent
8451        else:
8452            self.parent = None
8453        if not truncated and len(pages) == 1:
8454            s = product(pages[0].shape)
8455            if s > 0:
8456                self._len = int(product(self.shape) // s)
8457            else:
8458                self._len = len(pages)
8459        else:
8460            self._len = len(pages)
8461
8462    def set_shape_axes(self, shape, axes, squeeze=True):
8463        """Set shape and axes."""
8464        shape = tuple(shape)
8465        axes = ''.join(axes)
8466        # expanded shape according to metadata
8467        self._shape_expanded = shape
8468        self._axes_expanded = axes
8469        # squeezed shape and axes
8470        self._shape_squeezed, self._axes_squeezed = squeeze_axes(shape, axes)
8471        # default shape and axes returned by asarray
8472        self.shape = self._shape_squeezed if squeeze else self._shape_expanded
8473        self.axes = self._axes_squeezed if squeeze else self._axes_expanded
8474
8475    def get_shape(self, squeeze=None):
8476        """Return default, squeezed, or expanded shape."""
8477        if squeeze is None:
8478            return self.shape
8479        return self._shape_squeezed if squeeze else self._shape_expanded
8480
8481    def get_axes(self, squeeze=None):
8482        """Return default, squeezed, or expanded axes."""
8483        if squeeze is None:
8484            return self.axes
8485        return self._axes_squeezed if squeeze else self._axes_expanded
8486
8487    def asarray(self, level=None, **kwargs):
8488        """Return image data from series of TIFF pages as numpy array."""
8489        if level is not None:
8490            return self.levels[level].asarray(**kwargs)
8491        if self.parent:
8492            result = self.parent.asarray(series=self, **kwargs)
8493            if self.transform is not None:
8494                result = self.transform(result)
8495            return result
8496        return None
8497
8498    def aszarr(self, level=None, **kwargs):
8499        """Return image data from series of TIFF pages as zarr storage."""
8500        if self.parent:
8501            return ZarrTiffStore(self, level=level, **kwargs)
8502        return None
8503
8504    @lazyattr
8505    def offset(self):
8506        """Return offset to series data in file, if any."""
8507        if not self._pages:
8508            return None
8509
8510        pos = 0
8511        for page in self._pages:
8512            if page is None:
8513                return None
8514            if not page.is_final:
8515                return None
8516            if not pos:
8517                pos = page.is_contiguous[0] + page.is_contiguous[1]
8518                continue
8519            if pos != page.is_contiguous[0]:
8520                return None
8521            pos += page.is_contiguous[1]
8522
8523        page = self._pages[0]
8524        offset = page.is_contiguous[0]
8525        if (page.is_imagej or page.is_shaped or page.is_stk) and len(
8526            self._pages
8527        ) == 1:
8528            # truncated files
8529            return offset
8530        if pos == offset + product(self.shape) * self.dtype.itemsize:
8531            return offset
8532        return None
8533
8534    @property
8535    def is_pyramidal(self):
8536        """Return if series contains several levels."""
8537        return len(self.levels) > 1
8538
8539    @property
8540    def ndim(self):
8541        """Return number of array dimensions."""
8542        return len(self.shape)
8543
8544    @property
8545    def size(self):
8546        """Return number of elements in array."""
8547        return int(product(self.shape))
8548
8549    @property
8550    def pages(self):
8551        """Return sequence of all pages in series."""
8552        # a workaround to keep the old interface working
8553        return self
8554
8555    def _getitem(self, key):
8556        """Return specified page of series from cache or file."""
8557        key = int(key)
8558        if key < 0:
8559            key %= self._len
8560        if len(self._pages) == 1 and 0 < key < self._len:
8561            index = self._pages[0].index
8562            return self.parent.pages._getitem(index + key)
8563        return self._pages[key]
8564
8565    def __getitem__(self, key):
8566        """Return specified page(s)."""
8567        getitem = self._getitem
8568        if isinstance(key, (int, numpy.integer)):
8569            return getitem(key)
8570        if isinstance(key, slice):
8571            return [getitem(i) for i in range(*key.indices(self._len))]
8572        if isinstance(key, Iterable):
8573            return [getitem(k) for k in key]
8574        raise TypeError('key must be an integer, slice, or iterable')
8575
8576    def __iter__(self):
8577        """Return iterator over pages in series."""
8578        if len(self._pages) == self._len:
8579            yield from self._pages
8580        else:
8581            pages = self.parent.pages
8582            index = self._pages[0].index
8583            for i in range(self._len):
8584                yield pages[index + i]
8585
8586    def __len__(self):
8587        """Return number of pages in series."""
8588        return self._len
8589
8590    def __str__(self):
8591        """Return string with information about TiffPageSeries."""
8592        s = '  '.join(
8593            s
8594            for s in (
8595                snipstr(f'{self.name!r}', 20) if self.name else '',
8596                'x'.join(str(i) for i in self.shape),
8597                str(self.dtype),
8598                self.axes,
8599                self.kind,
8600                (f'{len(self.levels)} Levels') if self.is_pyramidal else '',
8601                f'{len(self)} Pages',
8602                (f'@{self.offset}') if self.offset else '',
8603            )
8604            if s
8605        )
8606        return f'TiffPageSeries {self.index}  {s}'
8607
8608
8609class ZarrStore(MutableMapping):
8610    """Zarr storage base class.
8611
8612    ZarrStore instances must be closed using the 'close' method, which is
8613    automatically called when using the 'with' context manager.
8614
8615    https://zarr.readthedocs.io/en/stable/spec/v2.html
8616    https://forum.image.sc/t/multiscale-arrays-v0-1/37930
8617
8618    """
8619
8620    def __init__(self, fillvalue=None, chunkmode=None):
8621        """Initialize ZarrStore."""
8622        self._store = {}
8623        self._fillvalue = 0 if fillvalue is None else fillvalue
8624        if chunkmode is None:
8625            self._chunkmode = TIFF.CHUNKMODE(0)
8626        else:
8627            self._chunkmode = enumarg(TIFF.CHUNKMODE, chunkmode)
8628
8629    def __enter__(self):
8630        return self
8631
8632    def __exit__(self, exc_type, exc_value, traceback):
8633        self.close()
8634
8635    def __del__(self):
8636        self.close()
8637
8638    def close(self):
8639        """Close ZarrStore."""
8640
8641    def flush(self):
8642        """Flush ZarrStore."""
8643        raise PermissionError('ZarrStore is read-only')
8644
8645    def clear(self):
8646        """Clear ZarrStore."""
8647        raise PermissionError('ZarrStore is read-only')
8648
8649    def keys(self):
8650        """Return keys in ZarrStore."""
8651        return self._store.keys()
8652
8653    def items(self):
8654        """Return items in ZarrStore."""
8655        return self._store.items()
8656
8657    def values(self):
8658        """Return values in ZarrStore."""
8659        return self._store.values()
8660
8661    def __iter__(self):
8662        return iter(self._store.keys())
8663
8664    def __len__(self):
8665        return len(self._store)
8666
8667    def __delitem__(self, key):
8668        raise PermissionError('ZarrStore is read-only')
8669
8670    def __contains__(self, key):
8671        return key in self._store
8672
8673    def __setitem__(self, key, value):
8674        raise PermissionError('ZarrStore is read-only')
8675
8676    def __getitem__(self, key):
8677        if key in self._store:
8678            return self._store[key]
8679        return self._getitem(key)
8680
8681    def _getitem(self, key):
8682        """Return chunk from file."""
8683        raise NotImplementedError
8684
8685    @property
8686    def is_multiscales(self):
8687        """Return if ZarrStore is multiscales."""
8688        return b'multiscales' in self._store['.zattrs']
8689
8690    @staticmethod
8691    def _empty_chunk(shape, dtype, fillvalue):
8692        """Return empty chunk."""
8693        if fillvalue is None or fillvalue == 0:
8694            return bytes(product(shape) * dtype.itemsize)
8695        chunk = numpy.empty(shape, dtype)
8696        chunk[:] = fillvalue
8697        return chunk  # .tobytes()
8698
8699    @staticmethod
8700    def _dtype(dtype):
8701        """Return dtype as string with native byte order."""
8702        if dtype.itemsize == 1:
8703            byteorder = '|'
8704        else:
8705            byteorder = {'big': '>', 'little': '<'}[sys.byteorder]
8706        return byteorder + dtype.str[1:]
8707
8708    @staticmethod
8709    def _json(obj):
8710        """Serialize obj to a JSON formatted string."""
8711        return json.dumps(
8712            obj,
8713            indent=1,
8714            sort_keys=True,
8715            ensure_ascii=True,
8716            separators=(',', ': '),
8717        ).encode('ascii')
8718
8719    @staticmethod
8720    def _value(value, dtype):
8721        """Return value which is serializable to JSON."""
8722        if value is None:
8723            return value
8724        if dtype.kind == 'b':
8725            return bool(value)
8726        if dtype.kind in 'ui':
8727            return int(value)
8728        if dtype.kind == 'f':
8729            if numpy.isnan(value):
8730                return 'NaN'
8731            if numpy.isposinf(value):
8732                return 'Infinity'
8733            if numpy.isneginf(value):
8734                return '-Infinity'
8735            return float(value)
8736        if dtype.kind in 'c':
8737            value = numpy.array(value, dtype)
8738            return (
8739                ZarrStore._value(value.real, dtype.type().real.dtype),
8740                ZarrStore._value(value.imag, dtype.type().imag.dtype),
8741            )
8742        return value
8743
8744    @staticmethod
8745    def _ndindex(shape, chunks):
8746        """Return iterator over all chunk index strings."""
8747        assert len(shape) == len(chunks)
8748        chunked = tuple(
8749            i // j + (1 if i % j else 0) for i, j in zip(shape, chunks)
8750        )
8751        for indices in numpy.ndindex(chunked):
8752            yield '.'.join(str(index) for index in indices)
8753
8754
8755class ZarrTiffStore(ZarrStore):
8756    """Zarr storage interface to image data in TiffPage or TiffPageSeries.
8757
8758    ZarrTiffStore instances are using a TiffFile instance for reading and
8759    decoding chunks. Therefore ZarrTiffStore instances cannot be pickled.
8760
8761    """
8762
8763    def __init__(
8764        self,
8765        arg,
8766        level=None,
8767        chunkmode=None,
8768        fillvalue=None,
8769        lock=None,
8770        squeeze=None,
8771        _openfiles=None,
8772    ):
8773        """Initialize Zarr storage from TiffPage or TiffPageSeries."""
8774        super().__init__(fillvalue=fillvalue, chunkmode=chunkmode)
8775
8776        if self._chunkmode not in (0, 2):
8777            raise NotImplementedError(f'{self._chunkmode!r} not implemented')
8778
8779        self._squeeze = None if squeeze is None else bool(squeeze)
8780        self._transform = getattr(arg, 'transform', None)
8781        self._data = getattr(arg, 'levels', [TiffPageSeries([arg])])
8782        if level is not None:
8783            self._data = [self._data[level]]
8784
8785        if lock is None:
8786            fh = self._data[0].keyframe.parent._parent.filehandle
8787            fh.lock = True
8788            lock = fh.lock
8789        self._filecache = FileCache(size=_openfiles, lock=lock)
8790
8791        if len(self._data) > 1:
8792            # multiscales
8793            self._store['.zgroup'] = ZarrStore._json({'zarr_format': 2})
8794            self._store['.zattrs'] = ZarrStore._json(
8795                {
8796                    'multiscales': [
8797                        {
8798                            'datasets': [
8799                                {'path': str(i)}
8800                                for i in range(len(self._data))
8801                            ],
8802                            'metadata': {},
8803                            'name': arg.name,
8804                            # 'type': 'gaussian',
8805                            'version': '0.1',
8806                        }
8807                    ]
8808                }
8809            )
8810            for level, series in enumerate(self._data):
8811                series.keyframe.decode  # cache decode function
8812                shape = series.get_shape(squeeze)
8813                dtype = series.dtype
8814                if fillvalue is None:
8815                    self._fillvalue = fillvalue = series.keyframe.nodata
8816                if self._chunkmode:
8817                    chunks = series.keyframe.shape
8818                else:
8819                    chunks = series.keyframe.chunks
8820                self._store[f'{level}/.zarray'] = ZarrStore._json(
8821                    {
8822                        'zarr_format': 2,
8823                        'shape': shape,
8824                        'chunks': ZarrTiffStore._chunks(chunks, shape),
8825                        'dtype': ZarrStore._dtype(dtype),
8826                        'compressor': None,
8827                        'fill_value': ZarrStore._value(fillvalue, dtype),
8828                        'order': 'C',
8829                        'filters': None,
8830                    }
8831                )
8832        else:
8833            series = self._data[0]
8834            series.keyframe.decode  # cache decode function
8835            shape = series.get_shape(squeeze)
8836            dtype = series.dtype
8837            if fillvalue is None:
8838                self._fillvalue = fillvalue = series.keyframe.nodata
8839            if self._chunkmode:
8840                chunks = series.keyframe.shape
8841            else:
8842                chunks = series.keyframe.chunks
8843            self._store['.zattrs'] = ZarrStore._json({})
8844            self._store['.zarray'] = ZarrStore._json(
8845                {
8846                    'zarr_format': 2,
8847                    'shape': shape,
8848                    'chunks': ZarrTiffStore._chunks(chunks, shape),
8849                    'dtype': ZarrStore._dtype(dtype),
8850                    'compressor': None,
8851                    'fill_value': ZarrStore._value(fillvalue, dtype),
8852                    'order': 'C',
8853                    'filters': None,
8854                }
8855            )
8856
8857    def close(self):
8858        """Close ZarrTiffStore."""
8859        if hasattr(self, '_filecache'):
8860            self._filecache.clear()
8861
8862    def write_fsspec(self, arg, url, compressors=None, version=None):
8863        """Write fsspec ReferenceFileSystem as JSON to file.
8864
8865        Url is the remote location of the TIFF file without the file name(s).
8866
8867        Raise ValueError if TIFF store can not be represented as
8868        ReferenceFileSystem due to features that are not supported by zarr
8869        or numcodecs:
8870
8871        * compressors, e.g. CCITT
8872        * filters, e.g. predictors, bitorder reversal, packed integers
8873        * dtypes, e.g. float24
8874        * JPEGTables in multi-page files
8875        * incomplete chunks, e.g. if imagelength % rowsperstrip != 0
8876
8877        Files containing incomplete tiles may fail at runtime.
8878
8879        https://github.com/intake/fsspec-reference-maker
8880
8881        """
8882        compressors = {
8883            1: None,
8884            8: 'zlib',
8885            32946: 'zlib',
8886            34925: 'lzma',
8887            50000: 'zstd',
8888            5: 'imagecodecs_lzw',
8889            7: 'imagecodecs_jpeg',
8890            22610: 'imagecodecs_jpegxr',  # NDPI
8891            32773: 'imagecodecs_packbits',
8892            33003: 'imagecodecs_jpeg2k',
8893            33004: 'imagecodecs_jpeg2k',
8894            33005: 'imagecodecs_jpeg2k',
8895            33007: 'imagecodecs_jpeg',  # ALT_JPG
8896            34712: 'imagecodecs_jpeg2k',
8897            34887: 'imagecodecs_lerc',
8898            34892: 'imagecodecs_jpeg',  # DNG lossy
8899            34933: 'imagecodecs_png',
8900            34934: 'imagecodecs_jpegxr',
8901            50001: 'imagecodecs_webp',
8902            50002: 'imagecodecs_jpegxl',
8903            **({} if compressors is None else compressors),
8904        }
8905
8906        for series in self._data:
8907            errormsg = ' not supported by the fsspec ReferenceFileSystem'
8908            keyframe = series.keyframe
8909            if keyframe.compression not in compressors:
8910                raise ValueError(f'{keyframe.compression!r} is' + errormsg)
8911            if keyframe.predictor != 1:
8912                raise ValueError(f'{keyframe.predictor!r} is' + errormsg)
8913            if keyframe.fillorder != 1:
8914                raise ValueError(f'{keyframe.fillorder!r} is' + errormsg)
8915            if keyframe.sampleformat not in (1, 2, 3, 6):
8916                raise ValueError(f'{keyframe.sampleformat!r} is' + errormsg)
8917            if keyframe.bitspersample not in (
8918                8,
8919                16,
8920                32,
8921                64,
8922                128,
8923            ) and keyframe.compression not in (
8924                7,
8925                33007,
8926                34892,
8927            ):  # JPEG
8928                raise ValueError(
8929                    f'BitsPerSample {keyframe.bitspersample} is' + errormsg
8930                )
8931            if (
8932                not self._chunkmode
8933                and not keyframe.is_tiled
8934                and keyframe.imagelength % keyframe.rowsperstrip
8935            ):
8936                raise ValueError('incomplete chunks are' + errormsg)
8937            if self._chunkmode and not keyframe.is_final:
8938                raise ValueError(f'{self._chunkmode!r} is' + errormsg)
8939            if keyframe.jpegtables is not None and len(series.pages) > 1:
8940                raise ValueError(
8941                    'JPEGTables in multi-page files are' + errormsg
8942                )
8943
8944        if url is None:
8945            url = ''
8946        elif url and url[-1] != '/':
8947            url += '/'
8948
8949        byteorder = '<' if sys.byteorder == 'big' else '>'
8950        if (
8951            self._data[0].keyframe.parent.byteorder != byteorder
8952            or self._data[0].keyframe.dtype.itemsize == 1
8953        ):
8954            byteorder = None
8955
8956        refs = dict()
8957        if version == 1:
8958            refs['version'] = 1
8959            refs['templates'] = {}
8960            refs['gen'] = []
8961            templates = {}
8962            if self._data[0].is_multifile:
8963                i = 0
8964                for page in self._data[0].pages:
8965                    if page is None:
8966                        continue
8967                    fname = page.keyframe.parent.filehandle.name
8968                    if fname in templates:
8969                        continue
8970                    key = f'u{i}'
8971                    templates[fname] = '{{%s}}' % key
8972                    refs['templates'][key] = url + fname
8973                    i += 1
8974            else:
8975                fname = self._data[0].keyframe.parent.filehandle.name
8976                key = 'u'
8977                templates[fname] = '{{%s}}' % key
8978                refs['templates'][key] = url + fname
8979
8980            refs['refs'] = refzarr = dict()
8981        else:
8982            refzarr = refs
8983
8984        for key, value in self._store.items():
8985            if '.zarray' in key:
8986                level = int(key.split('/')[0]) if '/' in key else 0
8987                keyframe = self._data[level].keyframe
8988                value = json.loads(value)
8989                codec_id = compressors[keyframe.compression]
8990                if codec_id == 'imagecodecs_jpeg':
8991                    # TODO: handle JPEG colorspaces
8992                    tables = keyframe.jpegtables
8993                    if tables is not None:
8994                        import base64
8995
8996                        tables = base64.b64encode(tables).decode()
8997                    header = keyframe.jpegheader
8998                    if header is not None:
8999                        import base64
9000
9001                        header = base64.b64encode(header).decode()
9002                    colorspace_jpeg, colorspace_data = jpeg_decode_colorspace(
9003                        keyframe.photometric,
9004                        keyframe.planarconfig,
9005                        keyframe.extrasamples,
9006                    )
9007                    value['compressor'] = {
9008                        'id': codec_id,
9009                        'tables': tables,
9010                        'header': header,
9011                        'bitspersample': keyframe.bitspersample,
9012                        'colorspace_jpeg': colorspace_jpeg,
9013                        'colorspace_data': colorspace_data,
9014                    }
9015                elif codec_id is not None:
9016                    value['compressor'] = {'id': codec_id}
9017                if byteorder is not None:
9018                    value['dtype'] = byteorder + value['dtype'][1:]
9019                value = ZarrStore._json(value)
9020
9021            refzarr[key] = value.decode()
9022
9023        if hasattr(arg, 'write'):
9024            fh = arg
9025        else:
9026            fh = open(arg, 'w')
9027
9028        if version == 1:
9029            fh.write(json.dumps(refs, indent=1).rsplit('}"', 1)[0] + '}"')
9030            indent = '  '
9031        else:
9032            fh.write(json.dumps(refs, indent=1)[:-2])
9033            indent = ' '
9034
9035        for key, value in self._store.items():
9036            if '.zarray' in key:
9037                value = json.loads(value)
9038                shape = value['shape']
9039                chunks = value['chunks']
9040                level = (key.split('/')[0] + '/') if '/' in key else ''
9041                for chunkindex in ZarrStore._ndindex(shape, chunks):
9042                    key = level + chunkindex
9043                    keyframe, page, _, offset, bytecount = self._parse_key(key)
9044                    if page and self._chunkmode and offset is None:
9045                        offset = page.dataoffsets[0]
9046                        bytecount = keyframe.nbytes
9047                    if offset and bytecount:
9048                        fname = keyframe.parent.filehandle.name
9049                        if version == 1:
9050                            fname = templates[fname]
9051                        else:
9052                            fname = f'{url}{fname}'
9053                        fh.write(
9054                            f',\n{indent}"{key}": '
9055                            f'["{fname}", {offset}, {bytecount}]'
9056                        )
9057
9058        if version == 1:
9059            fh.write('\n }')
9060        fh.write('\n}')
9061
9062        if not hasattr(arg, 'write'):
9063            fh.close()
9064
9065    def _getitem(self, key):
9066        """Return chunk from file."""
9067        keyframe, page, chunkindex, offset, bytecount = self._parse_key(key)
9068
9069        if self._chunkmode:
9070            chunks = keyframe.shape
9071        else:
9072            chunks = keyframe.chunks
9073
9074        if page is None or offset == 0 or bytecount == 0:
9075            chunk = ZarrStore._empty_chunk(
9076                chunks, keyframe.dtype, self._fillvalue
9077            )
9078            if self._transform is not None:
9079                chunk = self._transform(chunk)
9080            return chunk
9081
9082        if self._chunkmode and offset is None:
9083            chunk = page.asarray(lock=self._filecache.lock)  # maxworkers=1 ?
9084            if self._transform is not None:
9085                chunk = self._transform(chunk)
9086            return chunk
9087
9088        chunk = self._filecache.read(page.parent.filehandle, offset, bytecount)
9089
9090        decodeargs = {'_fullsize': True}
9091        if page.jpegtables is not None:
9092            decodeargs['jpegtables'] = page.jpegtables
9093        if keyframe.jpegheader is not None:
9094            decodeargs['jpegheader'] = keyframe.jpegheader
9095
9096        chunk = keyframe.decode(chunk, chunkindex, **decodeargs)[0]
9097        if self._transform is not None:
9098            chunk = self._transform(chunk)
9099
9100        if chunk.size != product(chunks):
9101            raise RuntimeError
9102        return chunk  # .tobytes()
9103
9104    def _parse_key(self, key):
9105        """Return keyframe, page, index, offset, and bytecount from key."""
9106        if len(self._data) > 1:
9107            # multiscales
9108            level, key = key.split('/')
9109            series = self._data[int(level)]
9110        else:
9111            series = self._data[0]
9112        keyframe = series.keyframe
9113        pageindex, chunkindex = self._indices(key, series)
9114        if pageindex > 0 and len(series) == 1:
9115            # truncated ImageJ, STK, or shaped
9116            if series.offset is None:
9117                raise RuntimeError('truncated series is not contiguous')
9118            page = series[0]
9119            if page is None:
9120                return keyframe, None, chunkindex, 0, 0
9121            offset = pageindex * page.size * page.dtype.itemsize
9122            offset += page.dataoffsets[chunkindex]
9123            if self._chunkmode:
9124                bytecount = page.size * page.dtype.itemsize
9125                return page.keyframe, page, chunkindex, offset, bytecount
9126        elif self._chunkmode:
9127            with self._filecache.lock:
9128                page = series[pageindex]
9129            if page is None:
9130                return keyframe, None, None, 0, 0
9131            return page.keyframe, page, None, None, None
9132        else:
9133            with self._filecache.lock:
9134                page = series[pageindex]
9135            if page is None:
9136                return keyframe, None, chunkindex, 0, 0
9137            offset = page.dataoffsets[chunkindex]
9138        bytecount = page.databytecounts[chunkindex]
9139        return page.keyframe, page, chunkindex, offset, bytecount
9140
9141    def _indices(self, key, series):
9142        """Return page and strile indices from zarr chunk index."""
9143        keyframe = series.keyframe
9144        shape = series.get_shape(self._squeeze)
9145        indices = [int(i) for i in key.split('.')]
9146        assert len(indices) == len(shape)
9147        if self._chunkmode:
9148            chunked = (1,) * len(keyframe.shape)
9149        else:
9150            chunked = keyframe.chunked
9151        p = 1
9152        for i, s in enumerate(shape[::-1]):
9153            p *= s
9154            if p == keyframe.size:
9155                i = len(indices) - i - 1
9156                frames_indices = indices[:i]
9157                strile_indices = indices[i:]
9158                frames_chunked = shape[:i]
9159                strile_chunked = list(shape[i:])  # updated later
9160                break
9161        else:
9162            raise RuntimeError
9163        if len(strile_chunked) == len(keyframe.shape):
9164            strile_chunked = chunked
9165        else:
9166            # get strile_chunked including singleton dimensions
9167            i = len(strile_indices) - 1
9168            j = len(keyframe.shape) - 1
9169            while True:
9170                if strile_chunked[i] == keyframe.shape[j]:
9171                    strile_chunked[i] = chunked[j]
9172                    i -= 1
9173                    j -= 1
9174                elif strile_chunked[i] == 1:
9175                    i -= 1
9176                else:
9177                    raise RuntimeError('shape does not match page shape')
9178                if i < 0 or j < 0:
9179                    break
9180            assert product(strile_chunked) == product(chunked)
9181        if len(frames_indices) > 0:
9182            frameindex = int(
9183                numpy.ravel_multi_index(frames_indices, frames_chunked)
9184            )
9185        else:
9186            frameindex = 0
9187        if len(strile_indices) > 0:
9188            strileindex = int(
9189                numpy.ravel_multi_index(strile_indices, strile_chunked)
9190            )
9191        else:
9192            strileindex = 0
9193        return frameindex, strileindex
9194
9195    @staticmethod
9196    def _chunks(chunks, shape):
9197        """Return chunks with same length as shape."""
9198        ndim = len(shape)
9199        if ndim == 0:
9200            return ()  # empty array
9201        if 0 in shape:
9202            return (1,) * ndim
9203        newchunks = []
9204        i = ndim - 1
9205        j = len(chunks) - 1
9206        while True:
9207            if j < 0:
9208                newchunks.append(1)
9209                i -= 1
9210            elif shape[i] > 1 and chunks[j] > 1:
9211                newchunks.append(chunks[j])
9212                i -= 1
9213                j -= 1
9214            elif shape[i] == chunks[j]:  # both 1
9215                newchunks.append(1)
9216                i -= 1
9217                j -= 1
9218            elif shape[i] == 1:
9219                newchunks.append(1)
9220                i -= 1
9221            elif chunks[j] == 1:
9222                newchunks.append(1)
9223                j -= 1
9224            else:
9225                raise RuntimeError
9226            if i < 0 or ndim == len(newchunks):
9227                break
9228        # assert ndim == len(newchunks)
9229        return tuple(newchunks[::-1])
9230
9231
9232class ZarrFileSequenceStore(ZarrStore):
9233    """Zarr storage interface to image data in FileSequence."""
9234
9235    def __init__(self, arg, fillvalue=None, chunkmode=None, **kwargs):
9236        """Initialize Zarr storage from FileSequence."""
9237        super().__init__(fillvalue=fillvalue, chunkmode=chunkmode)
9238
9239        if self._chunkmode not in (0, 3):
9240            raise NotImplementedError(f'{self._chunkmode!r} not implemented')
9241
9242        if not isinstance(arg, FileSequence):
9243            raise TypeError('not a FileSequence')
9244
9245        if arg._container:
9246            raise NotImplementedError('cannot open container as zarr storage')
9247
9248        image = arg.imread(arg.files[0], **kwargs)
9249        dtype = image.dtype
9250        chunks = image.shape
9251        shape = arg.shape + chunks
9252        chunks = (1,) * len(arg.shape) + chunks
9253
9254        self._store['.zattrs'] = ZarrStore._json({})
9255        self._store['.zarray'] = ZarrStore._json(
9256            {
9257                'zarr_format': 2,
9258                'shape': shape,
9259                'chunks': chunks,
9260                'dtype': ZarrStore._dtype(dtype),
9261                'compressor': None,
9262                'fill_value': ZarrStore._value(fillvalue, dtype),
9263                'order': 'C',
9264                'filters': None,
9265            }
9266        )
9267
9268        self._kwargs = kwargs
9269        self._imread = arg.imread
9270        self._lookup = dict(zip(arg.indices, arg.files))
9271        self._chunks = image.shape
9272        self._dtype = image.dtype
9273        self._cache = {arg.indices[0]: image}  # TODO: cache MRU number chunks
9274        self._commonpath = arg.commonpath()
9275
9276    def _getitem(self, key):
9277        """Return chunk from file."""
9278        indices = tuple(int(i) for i in key.split('.'))
9279        indices = indices[: -len(self._chunks)]
9280        if indices in self._cache:
9281            return self._cache[indices]
9282        self._cache.clear()
9283        filename = self._lookup.get(indices, None)
9284        if filename is None:
9285            chunk = ZarrStore._empty_chunk(
9286                self._chunks, self._dtype, self._fillvalue
9287            )
9288        else:
9289            chunk = self._imread(filename, **self._kwargs).tobytes()
9290        self._cache[indices] = chunk
9291        return chunk
9292
9293    def close(self):
9294        """Clear chunk cache."""
9295        self._cache.clear()
9296
9297    def write_fsspec(self, arg, url, codec_id=None, version=None):
9298        """Write fsspec ReferenceFileSystem as JSON to file.
9299
9300        Url is the remote location of the files without the file names.
9301
9302        """
9303        from urllib.parse import quote
9304
9305        kwargs = self._kwargs.copy()
9306
9307        if codec_id is not None:
9308            pass
9309        elif self._imread == imread:
9310            codec_id = 'tifffile'
9311        elif 'imagecodecs.' in self._imread.__module__:
9312            if (
9313                self._imread.__name__ != 'imread'
9314                or 'codec' not in self._kwargs
9315            ):
9316                raise ValueError('can not determine codec_id')
9317            codec = kwargs.pop('codec')
9318            if isinstance(codec, (list, tuple)):
9319                codec = codec[0]
9320            if callable(codec):
9321                codec = codec.__name__.split('_')[0]
9322            codec_id = {
9323                'avif': 'imagecodecs_avif',
9324                'gif': 'imagecodecs_gif',
9325                'jpeg': 'imagecodecs_jpeg',
9326                'jpeg8': 'imagecodecs_jpeg',
9327                'jpeg12': 'imagecodecs_jpeg',
9328                'jpeg2k': 'imagecodecs_jpeg2k',
9329                'jpegls': 'imagecodecs_jpegls',
9330                'jpegxl': 'imagecodecs_jpegxl',
9331                'jpegxr': 'imagecodecs_jpegxr',
9332                'ljpeg': 'imagecodecs_ljpeg',
9333                # 'npy': 'imagecodecs_npy',
9334                'png': 'imagecodecs_png',
9335                'tiff': 'imagecodecs_tiff',
9336                'webp': 'imagecodecs_webp',
9337                'zfp': 'imagecodecs_zfp',
9338            }[codec]
9339        else:
9340            raise ValueError('can not determine codec_id')
9341
9342        if url is None:
9343            url = ''
9344        elif url and url[-1] != '/':
9345            url += '/'
9346
9347        refs = dict()
9348        if version == 1:
9349            refs['version'] = 1
9350            refs['templates'] = {'u': url}
9351            refs['gen'] = []
9352            refs['refs'] = refzarr = dict()
9353            url = '{{u}}'
9354        else:
9355            refzarr = refs
9356
9357        for key, value in self._store.items():
9358            if '.zarray' in key:
9359                value = json.loads(value)
9360                # TODO: make kwargs serializable
9361                value['compressor'] = {'id': codec_id, **kwargs}
9362                value = ZarrStore._json(value)
9363            refzarr[key] = value.decode()
9364
9365        if hasattr(arg, 'write'):
9366            fh = arg
9367        else:
9368            fh = open(arg, 'w')
9369
9370        if version == 1:
9371            fh.write(json.dumps(refs, indent=1).rsplit('}"', 1)[0] + '}"')
9372            indent = '  '
9373        else:
9374            fh.write(json.dumps(refs, indent=1)[:-2])
9375            indent = ' '
9376
9377        prefix = len(self._commonpath)
9378
9379        for key, value in self._store.items():
9380            if '.zarray' in key:
9381                value = json.loads(value)
9382                shape = value['shape']
9383                chunks = value['chunks']
9384                for key in ZarrStore._ndindex(shape, chunks):
9385                    indices = tuple(int(i) for i in key.split('.'))
9386                    indices = indices[: -len(self._chunks)]
9387                    filename = self._lookup.get(indices, None)
9388                    if filename is not None:
9389                        filename = quote(filename[prefix:].replace('\\', '/'))
9390                        if filename[0] == '/':
9391                            filename = filename[1:]
9392                        fh.write(f',\n{indent}"{key}": ["{url}{filename}"]')
9393
9394        if version == 1:
9395            fh.write('\n }')
9396        fh.write('\n}')
9397
9398        if not hasattr(arg, 'write'):
9399            fh.close()
9400
9401
9402class FileSequence:
9403    """Series of files containing array data of compatible shape and data type.
9404
9405    Attributes
9406    ----------
9407    files : list
9408        List of file names.
9409    shape : tuple
9410        Shape of file series. Excludes shape of individual arrays.
9411    axes : str
9412        Labels of axes in shape.
9413    indices : tuple of tuples
9414        ND indices of files in shape.
9415
9416    """
9417
9418    def __init__(
9419        self,
9420        imread,
9421        files,
9422        container=None,
9423        sort=None,
9424        pattern=None,
9425        axesorder=None,
9426    ):
9427        r"""Initialize instance from multiple files.
9428
9429        Parameters
9430        ----------
9431        imread : function or class
9432            Array read function or class with asarray function returning numpy
9433            array from single file.
9434        files : str, path-like, or sequence thereof
9435            Glob filename pattern or sequence of file names. Default: \*.
9436            Binary streams are not supported.
9437        container : str or container instance
9438            Name or open instance of ZIP file in which files are stored.
9439        sort : function
9440            Sort function used to sort file names when 'files' is a pattern.
9441            The default (None) is natural_sorted. Use sort=False to disable
9442            sorting.
9443        pattern : str
9444            Regular expression pattern that matches axes and sequence indices
9445            in file names. By default (None), no pattern matching is performed.
9446            Axes can be specified by matching groups preceding the index groups
9447            in the file name, be provided as group names for the index groups,
9448            or be omitted. The predefined 'axes' pattern matches Olympus OIF
9449            and Leica TIFF series.
9450        axesorder : sequence of int
9451            Indices of axes in pattern.
9452
9453        """
9454        if files is None:
9455            files = '*'
9456        if sort is None:
9457            sort = natural_sorted
9458        self._container = container
9459        if container:
9460            import fnmatch
9461
9462            if isinstance(container, (str, os.PathLike)):
9463                import zipfile
9464
9465                self._container = zipfile.ZipFile(container)
9466            elif not hasattr(self._container, 'open'):
9467                raise ValueError('invalid container')
9468            if isinstance(files, str):
9469                files = fnmatch.filter(self._container.namelist(), files)
9470                if sort:
9471                    files = sort(files)
9472        elif isinstance(files, os.PathLike):
9473            files = [os.fspath(files)]
9474        elif isinstance(files, str):
9475            files = glob.glob(files)
9476            if sort:
9477                files = sort(files)
9478
9479        files = [os.fspath(f) for f in files]
9480        if not files:
9481            raise ValueError('no files found')
9482
9483        if hasattr(imread, 'asarray'):
9484            # redefine imread to use asarray from imread class
9485            if not callable(imread.asarray):
9486                raise ValueError('invalid imread function')
9487            _imread_ = imread
9488
9489            def imread(fname, **kwargs):
9490                with _imread_(fname) as handle:
9491                    return handle.asarray(**kwargs)
9492
9493        elif not callable(imread):
9494            raise ValueError('invalid imread function')
9495
9496        if container:
9497            # redefine imread to read from container
9498            _imread_ = imread
9499
9500            def imread(fname, **kwargs):
9501                with self._container.open(fname) as handle1:
9502                    with io.BytesIO(handle1.read()) as handle2:
9503                        return _imread_(handle2, **kwargs)
9504
9505        axes = 'I'
9506        shape = (len(files),)
9507        indices = tuple((i,) for i in range(len(files)))
9508        startindex = (0,)
9509
9510        pattern = TIFF.FILE_PATTERNS.get(pattern, pattern)
9511        if pattern:
9512            try:
9513                axes, shape, indices, startindex = parse_filenames(
9514                    files, pattern, axesorder
9515                )
9516            except ValueError as exc:
9517                log_warning(
9518                    f'FileSequence: failed to parse file names ({exc})'
9519                )
9520                axesorder = None
9521
9522        if product(shape) != len(files):
9523            log_warning(
9524                'FileSequence: files are missing. Missing data are zeroed'
9525            )
9526
9527        self.imread = imread
9528        self.files = files
9529        self.pattern = pattern
9530        self.axes = axes.upper()
9531        self.shape = shape
9532        self.indices = indices
9533        self._startindex = startindex
9534
9535    def __str__(self):
9536        """Return string with information about file FileSequence."""
9537        file = str(self._container) if self._container else self.files[0]
9538        file = os.path.split(file)[-1]
9539        return '\n '.join(
9540            (
9541                self.__class__.__name__,
9542                file,
9543                f'files: {len(self.files)}',
9544                'shape: {}'.format(', '.join(str(i) for i in self.shape)),
9545                f'axes: {self.axes}',
9546            )
9547        )
9548
9549    def __len__(self):
9550        return len(self.files)
9551
9552    def __enter__(self):
9553        return self
9554
9555    def __exit__(self, exc_type, exc_value, traceback):
9556        self.close()
9557
9558    def close(self):
9559        if self._container:
9560            self._container.close()
9561        self._container = None
9562
9563    def asarray(self, file=None, ioworkers=1, out=None, **kwargs):
9564        """Read image data from files and return as numpy array.
9565
9566        Raise IndexError or ValueError if array shapes do not match.
9567
9568        Parameters
9569        ----------
9570        file : int or None
9571            Index or name of single file to read.
9572        ioworkers : int or None
9573            Maximum number of threads to execute the array read function
9574            asynchronously. Default: 1.
9575            If None, default to the number of processors multiplied by 5.
9576            Using threads can significantly improve runtime when
9577            reading many small files from a network share.
9578        out : numpy.ndarray, str, or file-like object
9579            Buffer where image data are saved.
9580            If None (default), a new array is created.
9581            If numpy.ndarray, a writable array of compatible dtype and shape.
9582            If 'memmap', create a memory-mapped array in a temporary file.
9583            If str or open file, the file name or file object used to
9584            create a memory-map to an array stored in a binary file on disk.
9585        kwargs : dict
9586            Additional parameters passed to the array read function.
9587
9588        """
9589        if file is not None:
9590            if isinstance(file, int):
9591                return self.imread(self.files[file], **kwargs)
9592            return self.imread(file, **kwargs)
9593
9594        im = self.imread(self.files[0], **kwargs)
9595        shape = self.shape + im.shape
9596        result = create_output(out, shape, dtype=im.dtype)
9597        result = result.reshape(-1, *im.shape)
9598
9599        def func(index, fname):
9600            """Read single image from file into result."""
9601            index = int(numpy.ravel_multi_index(index, self.shape))
9602            im = self.imread(fname, **kwargs)
9603            result[index] = im
9604
9605        if len(self.files) < 2:
9606            ioworkers = 1
9607        elif ioworkers is None or ioworkers < 1:
9608            import multiprocessing
9609
9610            ioworkers = max(multiprocessing.cpu_count() * 5, 1)
9611
9612        if ioworkers < 2:
9613            for index, fname in zip(self.indices, self.files):
9614                func(index, fname)
9615        else:
9616            with ThreadPoolExecutor(ioworkers) as executor:
9617                for _ in executor.map(func, self.indices, self.files):
9618                    pass
9619
9620        result.shape = shape
9621        return result
9622
9623    def aszarr(self, **kwargs):
9624        """Return image data from files as zarr storage."""
9625        return ZarrFileSequenceStore(self, **kwargs)
9626
9627    def commonpath(self):
9628        """Return longest common sub-path of each file in sequence."""
9629        if len(self.files) == 1:
9630            commonpath = os.path.dirname(self.files[0])
9631        else:
9632            commonpath = os.path.commonpath(self.files)
9633        return commonpath
9634
9635
9636class TiffSequence(FileSequence):
9637    """Series of TIFF files."""
9638
9639    def __init__(
9640        self,
9641        files=None,
9642        container=None,
9643        sort=None,
9644        pattern=None,
9645        axesorder=None,
9646        imread=imread,
9647    ):
9648        """Initialize instance from multiple TIFF files."""
9649        super().__init__(
9650            imread,
9651            '*.tif' if files is None else files,
9652            container=container,
9653            sort=sort,
9654            pattern=pattern,
9655            axesorder=axesorder,
9656        )
9657
9658
9659class FileHandle:
9660    """Binary file handle.
9661
9662    A limited, special purpose file handle that can:
9663
9664    * handle embedded files (e.g. for LSM within LSM files)
9665    * re-open closed files (for multi-file formats, such as OME-TIFF)
9666    * read and write numpy arrays and records from file like objects
9667
9668    Only 'rb', 'r+b', and 'wb' modes are supported. Concurrently reading and
9669    writing of the same stream is untested.
9670
9671    When initialized from another file handle, do not use it unless this
9672    FileHandle is closed.
9673
9674    Attributes
9675    ----------
9676    name : str
9677        Name of the file.
9678    path : str
9679        Absolute path to file.
9680    size : int
9681        Size of file in bytes.
9682    is_file : bool
9683        If True, file has a fileno and can be memory-mapped.
9684
9685    All attributes are read-only.
9686
9687    """
9688
9689    __slots__ = (
9690        '_fh',
9691        '_file',
9692        '_mode',
9693        '_name',
9694        '_dir',
9695        '_lock',
9696        '_offset',
9697        '_size',
9698        '_close',
9699        'is_file',
9700    )
9701
9702    def __init__(self, file, mode=None, name=None, offset=None, size=None):
9703        """Initialize file handle from file name or another file handle.
9704
9705        Parameters
9706        ----------
9707        file : str, path-like, binary stream, or FileHandle
9708            File name or seekable binary stream, such as an open file
9709            or BytesIO.
9710        mode : str
9711            File open mode in case 'file' is a file name. Must be 'rb', 'r+b',
9712            or 'wb'. Default is 'rb'.
9713        name : str
9714            Optional name of file in case 'file' is a binary stream.
9715        offset : int
9716            Optional start position of embedded file. By default, this is
9717            the current file position.
9718        size : int
9719            Optional size of embedded file. By default, this is the number
9720            of bytes from the 'offset' to the end of the file.
9721
9722        """
9723        self._fh = None
9724        self._file = file
9725        self._mode = 'rb' if mode is None else mode
9726        self._name = name
9727        self._dir = ''
9728        self._offset = offset
9729        self._size = size
9730        self._close = True
9731        self.is_file = None
9732        self._lock = NullContext()
9733        self.open()
9734
9735    def open(self):
9736        """Open or re-open file."""
9737        if self._fh is not None:
9738            return  # file is open
9739
9740        if isinstance(self._file, os.PathLike):
9741            self._file = os.fspath(self._file)
9742        if isinstance(self._file, str):
9743            # file name
9744            self._file = os.path.realpath(self._file)
9745            self._dir, self._name = os.path.split(self._file)
9746            self._fh = open(self._file, self._mode)
9747            self._close = True
9748            if self._offset is None:
9749                self._offset = 0
9750        elif isinstance(self._file, FileHandle):
9751            # FileHandle
9752            self._fh = self._file._fh
9753            if self._offset is None:
9754                self._offset = 0
9755            self._offset += self._file._offset
9756            self._close = False
9757            if not self._name:
9758                if self._offset:
9759                    name, ext = os.path.splitext(self._file._name)
9760                    self._name = f'{name}@{self._offset}{ext}'
9761                else:
9762                    self._name = self._file._name
9763            if self._mode and self._mode != self._file._mode:
9764                raise ValueError('FileHandle has wrong mode')
9765            self._mode = self._file._mode
9766            self._dir = self._file._dir
9767        elif hasattr(self._file, 'seek'):
9768            # binary stream: open file, BytesIO
9769            try:
9770                self._file.tell()
9771            except Exception:
9772                raise ValueError('binary stream is not seekable')
9773            self._fh = self._file
9774            if self._offset is None:
9775                self._offset = self._file.tell()
9776            self._close = False
9777            if not self._name:
9778                try:
9779                    self._dir, self._name = os.path.split(self._fh.name)
9780                except AttributeError:
9781                    self._name = 'Unnamed binary stream'
9782            try:
9783                self._mode = self._fh.mode
9784            except AttributeError:
9785                pass
9786        else:
9787            raise ValueError(
9788                'the first parameter must be a file name, '
9789                'seekable binary stream, or FileHandle'
9790            )
9791
9792        if self._offset:
9793            self._fh.seek(self._offset)
9794
9795        if self._size is None:
9796            pos = self._fh.tell()
9797            self._fh.seek(self._offset, os.SEEK_END)
9798            self._size = self._fh.tell()
9799            self._fh.seek(pos)
9800
9801        if self.is_file is None:
9802            try:
9803                self._fh.fileno()
9804                self.is_file = True
9805            except Exception:
9806                self.is_file = False
9807
9808    def close(self):
9809        """Close file."""
9810        if self._close and self._fh:
9811            self._fh.close()
9812            self._fh = None
9813
9814    def tell(self):
9815        """Return file's current position."""
9816        return self._fh.tell() - self._offset
9817
9818    def seek(self, offset, whence=0):
9819        """Set file's current position."""
9820        if self._offset:
9821            if whence == 0:
9822                self._fh.seek(self._offset + offset, whence)
9823                return
9824            if whence == 2 and self._size > 0:
9825                self._fh.seek(self._offset + self._size + offset, 0)
9826                return
9827        self._fh.seek(offset, whence)
9828
9829    def read(self, size=-1):
9830        """Read 'size' bytes from file, or until EOF is reached."""
9831        if size < 0 and self._offset:
9832            size = self._size
9833        return self._fh.read(size)
9834
9835    def readinto(self, b):
9836        """Read up to len(b) bytes into b and return number of bytes read."""
9837        return self._fh.readinto(b)
9838
9839    def write(self, bytestring):
9840        """Write bytes to file."""
9841        return self._fh.write(bytestring)
9842
9843    def flush(self):
9844        """Flush write buffers if applicable."""
9845        return self._fh.flush()
9846
9847    def memmap_array(self, dtype, shape, offset=0, mode='r', order='C'):
9848        """Return numpy.memmap of data stored in file."""
9849        if not self.is_file:
9850            raise ValueError('cannot memory-map file without fileno')
9851        return numpy.memmap(
9852            self._fh,
9853            dtype=dtype,
9854            mode=mode,
9855            offset=self._offset + offset,
9856            shape=shape,
9857            order=order,
9858        )
9859
9860    def read_array(self, dtype, count=-1, out=None):
9861        """Return numpy array from file in native byte order."""
9862        dtype = numpy.dtype(dtype)
9863
9864        if count < 0:
9865            nbytes = self._size if out is None else out.nbytes
9866            count = nbytes // dtype.itemsize
9867        else:
9868            nbytes = count * dtype.itemsize
9869
9870        result = numpy.empty(count, dtype) if out is None else out
9871
9872        if result.nbytes != nbytes:
9873            raise ValueError('size mismatch')
9874
9875        try:
9876            n = self._fh.readinto(result)
9877        except AttributeError:
9878            result[:] = numpy.frombuffer(self._fh.read(nbytes), dtype).reshape(
9879                result.shape
9880            )
9881            n = nbytes
9882
9883        if n != nbytes:
9884            raise ValueError(f'failed to read {nbytes} bytes')
9885
9886        if not result.dtype.isnative:
9887            if not dtype.isnative:
9888                result.byteswap(True)
9889            result = result.newbyteorder()
9890        elif result.dtype.isnative != dtype.isnative:
9891            result.byteswap(True)
9892
9893        if out is not None:
9894            if hasattr(out, 'flush'):
9895                out.flush()
9896
9897        return result
9898
9899    def read_record(self, dtype, shape=1, byteorder=None):
9900        """Return numpy record from file."""
9901        rec = numpy.rec
9902        try:
9903            record = rec.fromfile(self._fh, dtype, shape, byteorder=byteorder)
9904        except Exception:
9905            dtype = numpy.dtype(dtype)
9906            if shape is None:
9907                shape = self._size // dtype.itemsize
9908            size = product(sequence(shape)) * dtype.itemsize
9909            # data = bytearray(size)
9910            # n = self._fh.readinto(data)
9911            # data = data[:n]
9912            # TODO: record is not writable
9913            data = self._fh.read(size)
9914            record = rec.fromstring(data, dtype, shape, byteorder=byteorder)
9915        return record[0] if shape == 1 else record
9916
9917    def write_empty(self, size):
9918        """Append size bytes to file. Position must be at end of file."""
9919        if size < 1:
9920            return
9921        self._fh.seek(size - 1, os.SEEK_CUR)
9922        self._fh.write(b'\x00')
9923
9924    def write_array(self, data):
9925        """Write numpy array to binary file."""
9926        try:
9927            # writing non-contiguous arrays is very slow
9928            numpy.ascontiguousarray(data).tofile(self._fh)
9929        except Exception:
9930            # numpy can't write to BytesIO
9931            self._fh.write(data.tobytes())
9932
9933    def read_segments(
9934        self,
9935        offsets,
9936        bytecounts,
9937        indices=None,
9938        sort=True,
9939        lock=None,
9940        buffersize=None,
9941        flat=True,
9942    ):
9943        """Return iterator over segments read from file and their indices.
9944
9945        The purpose of this function is to
9946
9947        * reduce small or random reads
9948        * reduce acquiring reentrant locks
9949        * synchronize seeks and reads
9950        * limit the size of segments read into memory at once
9951          (ThreadPoolExecutor.map is not collecting iterables lazily).
9952
9953        Parameters
9954        ----------
9955        offsets, bytecounts : sequence of int
9956            offsets and bytecounts of the segments to read from file.
9957        indices : sequence of int
9958            Indices of the segments in the image. Default: range(len(offsets)).
9959        sort : bool
9960            If True (default), segments are read from file in the order of
9961            their offsets.
9962        lock:
9963            A reentrant lock used to synchronize seeks and reads.
9964        buffersize : int
9965            Approximate number of bytes to read from file in one pass.
9966            Default: 64 MB.
9967        flat : bool
9968            If True (default), return an iterator over individual
9969            (segment, index) tuples. Else return an iterator over a list
9970            of (segment, index) tuples that were acquired in one pass.
9971
9972        Returns
9973        -------
9974        items : (bytes, int) or [(bytes, int)]
9975            Iterator over individual or lists of (segment, index) tuples.
9976
9977        """
9978        # TODO: Cythonize this?
9979        length = len(offsets)
9980        if length < 1:
9981            return
9982        if length == 1:
9983            index = 0 if indices is None else indices[0]
9984            if bytecounts[index] > 0 and offsets[index] > 0:
9985                if lock is None:
9986                    lock = self._lock
9987                with lock:
9988                    self.seek(offsets[index])
9989                    data = self._fh.read(bytecounts[index])
9990            else:
9991                data = None
9992            yield (data, index) if flat else [(data, index)]
9993            return
9994
9995        if lock is None:
9996            lock = self._lock
9997        if buffersize is None:
9998            buffersize = 67108864  # 2 ** 26, 64 MB
9999
10000        if indices is None:
10001            segments = [(i, offsets[i], bytecounts[i]) for i in range(length)]
10002        else:
10003            segments = [
10004                (indices[i], offsets[i], bytecounts[i]) for i in range(length)
10005            ]
10006        if sort:
10007            segments = sorted(segments, key=lambda x: x[1])
10008
10009        iscontig = True
10010        for i in range(length - 1):
10011            _, offset, bytecount = segments[i]
10012            nextoffset = segments[i + 1][1]
10013            if offset == 0 or bytecount == 0 or nextoffset == 0:
10014                continue
10015            if offset + bytecount != nextoffset:
10016                iscontig = False
10017                break
10018
10019        seek = self.seek
10020        read = self._fh.read
10021
10022        if iscontig:
10023            # consolidate reads
10024            i = 0
10025            while i < length:
10026                j = i
10027                offset = None
10028                bytecount = 0
10029                while bytecount < buffersize and i < length:
10030                    _, o, b = segments[i]
10031                    if o > 0 and b > 0:
10032                        if offset is None:
10033                            offset = o
10034                        bytecount += b
10035                    i += 1
10036
10037                if offset is None:
10038                    data = None
10039                else:
10040                    with lock:
10041                        seek(offset)
10042                        data = read(bytecount)
10043                start = 0
10044                stop = 0
10045                result = []
10046                while j < i:
10047                    index, offset, bytecount = segments[j]
10048                    if offset > 0 and bytecount > 0:
10049                        stop += bytecount
10050                        result.append((data[start:stop], index))
10051                        start = stop
10052                    else:
10053                        result.append((None, index))
10054                    j += 1
10055                if flat:
10056                    yield from result
10057                else:
10058                    yield result
10059            return
10060
10061        i = 0
10062        while i < length:
10063            result = []
10064            size = 0
10065            with lock:
10066                while size < buffersize and i < length:
10067                    index, offset, bytecount = segments[i]
10068                    if offset > 0 and bytecount > 0:
10069                        seek(offset)
10070                        result.append((read(bytecount), index))
10071                        # buffer = bytearray(bytecount)
10072                        # n = fh.readinto(buffer)
10073                        # data.append(buffer[:n])
10074                        size += bytecount
10075                    else:
10076                        result.append((None, index))
10077                    i += 1
10078            if flat:
10079                yield from result
10080            else:
10081                yield result
10082
10083    def __enter__(self):
10084        return self
10085
10086    def __exit__(self, exc_type, exc_value, traceback):
10087        self.close()
10088
10089    def __getattr__(self, name):
10090        """Return attribute from underlying file object."""
10091        if self._offset:
10092            warnings.warn(
10093                f'FileHandle: {name!r} not implemented for embedded files',
10094                UserWarning,
10095            )
10096        return getattr(self._fh, name)
10097
10098    @property
10099    def name(self):
10100        return self._name
10101
10102    @property
10103    def dirname(self):
10104        return self._dir
10105
10106    @property
10107    def path(self):
10108        return os.path.join(self._dir, self._name)
10109
10110    @property
10111    def size(self):
10112        return self._size
10113
10114    @property
10115    def closed(self):
10116        return self._fh is None
10117
10118    @property
10119    def lock(self):
10120        """Return current lock instance."""
10121        return self._lock
10122
10123    @lock.setter
10124    def lock(self, value):
10125        if bool(value) == isinstance(self._lock, NullContext):
10126            self._lock = threading.RLock() if value else NullContext()
10127
10128    @property
10129    def has_lock(self):
10130        """Return if a RLock is used."""
10131        return not isinstance(self._lock, NullContext)
10132
10133
10134class FileCache:
10135    """Keep FileHandles open."""
10136
10137    __slots__ = ('files', 'keep', 'past', 'lock', 'size')
10138
10139    def __init__(self, size=None, lock=None):
10140        """Initialize open file cache."""
10141        self.past = []  # FIFO of opened files
10142        self.files = {}  # refcounts of opened file handles
10143        self.keep = set()  # files to keep open
10144        self.lock = NullContext() if lock is None else lock
10145        self.size = 8 if size is None else int(size)
10146
10147    def __len__(self):
10148        """Return number of open files."""
10149        return len(self.files)
10150
10151    def open(self, filehandle):
10152        """Open file, re-open if necessary."""
10153        with self.lock:
10154            if filehandle in self.files:
10155                self.files[filehandle] += 1
10156            elif filehandle.closed:
10157                filehandle.open()
10158                self.files[filehandle] = 1
10159                self.past.append(filehandle)
10160            else:
10161                self.files[filehandle] = 2
10162                self.keep.add(filehandle)
10163                self.past.append(filehandle)
10164
10165    def close(self, filehandle):
10166        """Close least recently used open files."""
10167        with self.lock:
10168            if filehandle in self.files:
10169                self.files[filehandle] -= 1
10170            self._trim()
10171
10172    def clear(self):
10173        """Close all opened files if not in use when opened first."""
10174        with self.lock:
10175            for filehandle, refcount in list(self.files.items()):
10176                if filehandle not in self.keep:
10177                    filehandle.close()
10178                    del self.files[filehandle]
10179                    del self.past[self.past.index(filehandle)]
10180
10181    def read(self, filehandle, offset, bytecount, whence=0):
10182        """Return bytes read from binary file."""
10183        with self.lock:
10184            b = filehandle not in self.files
10185            if b:
10186                if filehandle.closed:
10187                    filehandle.open()
10188                    self.files[filehandle] = 0
10189                else:
10190                    self.files[filehandle] = 1
10191                    self.keep.add(filehandle)
10192                self.past.append(filehandle)
10193            filehandle.seek(offset, whence)
10194            data = filehandle.read(bytecount)
10195            if b:
10196                self._trim()
10197        return data
10198
10199    def _trim(self):
10200        """Trim the file cache."""
10201        index = 0
10202        size = len(self.past)
10203        while size > self.size and index < size:
10204            filehandle = self.past[index]
10205            if filehandle not in self.keep and self.files[filehandle] <= 0:
10206                filehandle.close()
10207                del self.files[filehandle]
10208                del self.past[index]
10209                size -= 1
10210            else:
10211                index += 1
10212
10213
10214class NullContext:
10215    """Null context manager.
10216
10217    >>> with NullContext():
10218    ...     pass
10219
10220    """
10221
10222    __slots = ()
10223
10224    def __enter__(self):
10225        return self
10226
10227    def __exit__(self, exc_type, exc_value, traceback):
10228        pass
10229
10230
10231class Timer:
10232    """Stopwatch for timing execution speed."""
10233
10234    __slots__ = ('started', 'stopped', 'duration')
10235
10236    clock = time.perf_counter
10237
10238    def __init__(self, message=None, end=' '):
10239        """Initialize timer and print message."""
10240        if message is not None:
10241            print(message, end=end, flush=True)
10242        self.duration = 0
10243        self.started = self.stopped = Timer.clock()
10244
10245    def start(self, message=None, end=' '):
10246        """Start timer and return current time."""
10247        if message is not None:
10248            print(message, end=end, flush=True)
10249        self.duration = 0
10250        self.started = self.stopped = Timer.clock()
10251        return self.started
10252
10253    def stop(self, message=None, end=' '):
10254        """Return duration of timer till start."""
10255        self.stopped = Timer.clock()
10256        if message is not None:
10257            print(message, end=end, flush=True)
10258        self.duration = self.stopped - self.started
10259        return self.duration
10260
10261    def print(self, message=None, end=None):
10262        """Print duration from timer start till last stop or now."""
10263        msg = str(self)
10264        if message is not None:
10265            print(message, end=' ')
10266        print(msg, end=end, flush=True)
10267
10268    def __str__(self):
10269        """Return duration from timer start till last stop or now as string."""
10270        if self.duration <= 0:
10271            # not stopped
10272            duration = Timer.clock() - self.started
10273        else:
10274            duration = self.duration
10275        s = str(datetime.timedelta(seconds=duration))
10276        i = 0
10277        while i < len(s) and s[i : i + 2] in '0:0010203040506070809':
10278            i += 1
10279        if s[i : i + 1] == ':':
10280            i += 1
10281        return f'{s[i:]} s'
10282
10283    def __enter__(self):
10284        return self
10285
10286    def __exit__(self, exc_type, exc_value, traceback):
10287        self.print()
10288
10289
10290class OmeXmlError(Exception):
10291    """Exception to indicate invalid OME-XML or unsupported cases."""
10292
10293
10294class OmeXml:
10295    """OME-TIFF XML."""
10296
10297    def __init__(self, **metadata):
10298        """Create a new instance.
10299
10300        Creator : str (optional)
10301            Name of the creating application. Default 'tifffile.py'.
10302        UUID : str (optional)
10303            Unique identifier.
10304
10305        """
10306        if 'OME' in metadata:
10307            metadata = metadata['OME']
10308
10309        self.ifd = 0
10310        self.images = []
10311        self.annotations = []
10312        self.elements = []
10313        # TODO: parse other OME elements from metadata
10314        #   Project
10315        #   Dataset
10316        #   Folder
10317        #   Experiment
10318        #   Plate
10319        #   Screen
10320        #   Experimenter
10321        #   ExperimenterGroup
10322        #   Instrument
10323        #   StructuredAnnotations
10324        #   ROI
10325        if 'UUID' in metadata:
10326            self.uuid = metadata['UUID'].split(':')[-1]
10327        else:
10328            from uuid import uuid1  # noqa: delayed import
10329
10330            self.uuid = str(uuid1())
10331        creator = OmeXml._attribute(
10332            metadata, 'Creator', default=f'tifffile.py {__version__}'
10333        )
10334        schema = 'http://www.openmicroscopy.org/Schemas/OME/2016-06'
10335        self.xml = (
10336            '{declaration}'
10337            f'<OME xmlns="{schema}" '
10338            f'xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" '
10339            f'xsi:schemaLocation="{schema} {schema}/ome.xsd" '
10340            f'UUID="urn:uuid:{self.uuid}" {creator}>'
10341            '{images}'
10342            '{annotations}'
10343            '{elements}'
10344            f'</OME>'
10345        )
10346
10347    def addimage(self, dtype, shape, storedshape, axes=None, **metadata):
10348        """Add image to OME-XML.
10349
10350        The OME model can handle up to 9 dimensional images for selected
10351        axes orders. Refer to the OME-XML specification for details.
10352        Non-TZCYXS (modulo) dimensions must be after a TZC dimension or
10353        require an unused TZC dimension.
10354
10355        Parameters
10356        ----------
10357        dtype : numpy.dtype
10358            Data type of image array.
10359        shape : tuple
10360            Shape of image array.
10361        storedshape: tuple
10362            Normalized shape describing how the image array is stored in TIFF:
10363            (pages, separate_samples, depth, length, width, contig_samples).
10364        axes : str (optional)
10365            Axes labels for each dimension in shape.
10366            By default, axes are matched to the shape in reverse order of
10367            TZC(S)YX(S) based on storedshape.
10368            The following axes codes are supported: 'S' sample, 'X' width,
10369            'Y' length, 'Z' depth, 'C' channel, 'T' time, 'A' angle, 'P' phase,
10370            'R' tile, 'H' lifetime, 'E' lambda, 'Q' other.
10371        metadata : miscellaneous (optional)
10372            Additional OME-XML attributes or elements to be stored.
10373            Image/Pixels: Name, AcquisitionDate, Description,
10374            PhysicalSizeX, PhysicalSizeXUnit, PhysicalSizeY, PhysicalSizeYUnit,
10375            PhysicalSizeZ, PhysicalSizeZUnit, TimeIncrement, TimeIncrementUnit.
10376            Per Plane: DeltaTUnit, ExposureTime, ExposureTimeUnit,
10377            PositionX, PositionXUnit, PositionY, PositionYUnit, PositionZ,
10378            PositionZUnit.
10379            Per Channel: Name, AcquisitionMode, Color, ContrastMethod,
10380            EmissionWavelength, EmissionWavelengthUnit, ExcitationWavelength,
10381            ExcitationWavelengthUnit, Fluor, IlluminationType, NDFilter,
10382            PinholeSize, PinholeSizeUnit, PockelCellSetting.
10383
10384        """
10385        index = len(self.images)
10386
10387        # get Image and Pixels metadata
10388        metadata = metadata.get('OME', metadata)
10389        metadata = metadata.get('Image', metadata)
10390        if isinstance(metadata, (list, tuple)):
10391            # multiple images
10392            metadata = metadata[index]
10393        if 'Pixels' in metadata:
10394            # merge with Image
10395            if 'ID' in metadata['Pixels']:
10396                del metadata['Pixels']['ID']
10397            metadata.update(metadata['Pixels'])
10398            del metadata['Pixels']
10399
10400        try:
10401            dtype = numpy.dtype(dtype).name
10402            dtype = {
10403                'int8': 'int8',
10404                'int16': 'int16',
10405                'int32': 'int32',
10406                'uint8': 'uint8',
10407                'uint16': 'uint16',
10408                'uint32': 'uint32',
10409                'float32': 'float',
10410                'float64': 'double',
10411                'complex64': 'complex',
10412                'complex128': 'double-complex',
10413                'bool': 'bit',
10414            }[dtype]
10415        except KeyError:
10416            raise OmeXmlError(f'data type {dtype!r} not supported')
10417
10418        if metadata.get('Type', dtype) != dtype:
10419            raise OmeXmlError(
10420                f'metadata Pixels Type {metadata["Type"]!r} '
10421                f'does not match array dtype {dtype!r}'
10422            )
10423
10424        samples = 1
10425        planecount, separate, depth, length, width, contig = storedshape
10426        if depth != 1:
10427            raise OmeXmlError('ImageDepth not supported')
10428        if not (separate == 1 or contig == 1):
10429            raise ValueError('invalid stored shape')
10430
10431        shape = tuple(int(i) for i in shape)
10432        ndim = len(shape)
10433        if ndim < 1 or product(shape) <= 0:
10434            raise OmeXmlError('empty arrays not supported')
10435
10436        if axes is None:
10437            # get axes from shape, stored shape, and DimensionOrder
10438            if contig != 1 or shape[-3:] == (length, width, 1):
10439                axes = 'YXS'
10440                samples = contig
10441            elif separate != 1 or (
10442                ndim == 6 and shape[-3:] == (1, length, width)
10443            ):
10444                axes = 'SYX'
10445                samples = separate
10446            else:
10447                axes = 'YX'
10448            if not len(axes) <= ndim <= (6 if 'S' in axes else 5):
10449                raise OmeXmlError(f'{ndim} dimensions not supported')
10450            hiaxes = metadata.get('DimensionOrder', 'XYCZT')[:1:-1]
10451            axes = hiaxes[(6 if 'S' in axes else 5) - ndim :] + axes
10452            assert len(axes) == len(shape)
10453
10454        else:
10455            # validate axes against shape and stored shape
10456            axes = axes.upper()
10457            if len(axes) != len(shape):
10458                raise ValueError('axes do not match shape')
10459            if not (axes.endswith('YX') or axes.endswith('YXS')):
10460                raise OmeXmlError('dimensions must end with YX or YXS')
10461            unique = []
10462            for ax in axes:
10463                if ax not in 'TZCYXSAPRHEQ':
10464                    raise OmeXmlError(f'dimension {ax!r} not supported')
10465                if ax in unique:
10466                    raise OmeXmlError(f'multiple {ax!r} dimensions')
10467                unique.append(ax)
10468            if ndim > (9 if 'S' in axes else 8):
10469                raise OmeXmlError('more than 8 dimensions not supported')
10470            if contig != 1:
10471                samples = contig
10472                if ndim < 3:
10473                    raise ValueError('dimensions do not match stored shape')
10474                if axes[-1] != 'S':
10475                    raise ValueError('axes do not match stored shape')
10476                if shape[-1] != contig or shape[-2] != width:
10477                    raise ValueError('shape does not match stored shape')
10478            elif separate != 1:
10479                samples = separate
10480                if ndim < 3:
10481                    raise ValueError('dimensions do not match stored shape')
10482                if axes[-3] != 'S':
10483                    raise ValueError('axes do not match stored shape')
10484                if shape[-3] != separate or shape[-1] != length:
10485                    raise ValueError('shape does not match stored shape')
10486
10487        if shape[axes.index('X')] != width or shape[axes.index('Y')] != length:
10488            raise ValueError('shape does not match stored shape')
10489
10490        if 'S' in axes:
10491            hiaxes = axes[: min(axes.index('S'), axes.index('Y'))]
10492        else:
10493            hiaxes = axes[: axes.index('Y')]
10494
10495        if any(ax in 'APRHEQ' for ax in hiaxes):
10496            # modulo axes
10497            modulo = {}
10498            dimorder = []
10499            axestype = {
10500                'A': 'angle',
10501                'P': 'phase',
10502                'R': 'tile',
10503                'H': 'lifetime',
10504                'E': 'lambda',
10505                'Q': 'other',
10506            }
10507            for i, ax in enumerate(hiaxes):
10508                if ax in 'APRHEQ':
10509                    x = hiaxes[i - 1 : i]
10510                    if x and x in 'TZC':
10511                        # use previous axis
10512                        modulo[x] = axestype[ax], shape[i]
10513                    else:
10514                        # use next unused axis
10515                        for x in 'TZC':
10516                            if x not in dimorder and x not in modulo:
10517                                modulo[x] = axestype[ax], shape[i]
10518                                dimorder.append(x)
10519                                break
10520                        else:
10521                            # TODO: support any order of axes, e.g. APRTZC
10522                            raise OmeXmlError('more than 3 modulo dimensions')
10523                else:
10524                    dimorder.append(ax)
10525            hiaxes = ''.join(dimorder)
10526
10527            # TODO: use user-specified start, stop, step, or labels
10528            moduloalong = ''.join(
10529                f'<ModuloAlong{ax} Type="{axtype}" Start="0" End="{size-1}"/>'
10530                for ax, (axtype, size) in modulo.items()
10531            )
10532            annotationref = f'<AnnotationRef ID="Annotation:{index}"/>'
10533            annotations = (
10534                f'<XMLAnnotation ID="Annotation:{index}" '
10535                'Namespace="openmicroscopy.org/omero/dimension/modulo">'
10536                '<Value>'
10537                '<Modulo namespace='
10538                '"http://www.openmicroscopy.org/Schemas/Additions/2011-09">'
10539                f'{moduloalong}'
10540                '</Modulo>'
10541                '</Value>'
10542                '</XMLAnnotation>'
10543            )
10544            self.annotations.append(annotations)
10545        else:
10546            modulo = {}
10547            annotationref = ''
10548
10549        hiaxes = hiaxes[::-1]
10550        for dimorder in (
10551            metadata.get('DimensionOrder', 'XYCZT'),
10552            'XYCZT',
10553            'XYZCT',
10554            'XYZTC',
10555            'XYCTZ',
10556            'XYTCZ',
10557            'XYTZC',
10558        ):
10559            if hiaxes in dimorder:
10560                break
10561        else:
10562            raise OmeXmlError(f'dimension order {axes!r} not supported')
10563
10564        dimsizes = []
10565        for ax in dimorder:
10566            if ax == 'S':
10567                continue
10568            if ax in axes:
10569                size = shape[axes.index(ax)]
10570            else:
10571                size = 1
10572            if ax == 'C':
10573                sizec = size
10574                size *= samples
10575            if ax in modulo:
10576                size *= modulo[ax][1]
10577            dimsizes.append(size)
10578        sizes = ''.join(
10579            f' Size{ax}="{size}"' for ax, size in zip(dimorder, dimsizes)
10580        )
10581
10582        # verify DimensionOrder in metadata is compatible
10583        if 'DimensionOrder' in metadata:
10584            omedimorder = metadata['DimensionOrder']
10585            omedimorder = ''.join(
10586                ax for ax in omedimorder if dimsizes[dimorder.index(ax)] > 1
10587            )
10588            if hiaxes not in omedimorder:
10589                raise OmeXmlError(
10590                    f'metadata DimensionOrder does not match {axes!r}'
10591                )
10592
10593        # verify metadata Size values match shape
10594        for ax, size in zip(dimorder, dimsizes):
10595            if metadata.get(f'Size{ax}', size) != size:
10596                raise OmeXmlError(
10597                    f'metadata Size{ax} does not match {shape!r}'
10598                )
10599
10600        dimsizes[dimorder.index('C')] //= samples
10601        if planecount != product(dimsizes[2:]):
10602            raise ValueError('shape does not match stored shape')
10603
10604        planes = []
10605        planeattributes = metadata.get('Plane', '')
10606        if planeattributes:
10607            cztorder = tuple(dimorder[2:].index(ax) for ax in 'CZT')
10608            for p in range(planecount):
10609                attributes = OmeXml._attributes(
10610                    planeattributes,
10611                    p,
10612                    'DeltaTUnit',
10613                    'ExposureTime',
10614                    'ExposureTimeUnit',
10615                    'PositionX',
10616                    'PositionXUnit',
10617                    'PositionY',
10618                    'PositionYUnit',
10619                    'PositionZ',
10620                    'PositionZUnit',
10621                )
10622                unraveled = numpy.unravel_index(p, dimsizes[2:], order='F')
10623                c, z, t = (int(unraveled[i]) for i in cztorder)
10624                planes.append(
10625                    f'<Plane TheC="{c}" TheZ="{z}" TheT="{t}"{attributes}/>'
10626                )
10627                # TODO: if possible, verify c, z, t match planeattributes
10628        planes = ''.join(planes)
10629
10630        channels = []
10631        for c in range(sizec):
10632            lightpath = '<LightPath/>'
10633            # TODO: use LightPath elements from metadata
10634            #    'AnnotationRef',
10635            #    'DichroicRef',
10636            #    'EmissionFilterRef',
10637            #    'ExcitationFilterRef'
10638            attributes = OmeXml._attributes(
10639                metadata.get('Channel', ''),
10640                c,
10641                'Name',
10642                'AcquisitionMode',
10643                'Color',
10644                'ContrastMethod',
10645                'EmissionWavelength',
10646                'EmissionWavelengthUnit',
10647                'ExcitationWavelength',
10648                'ExcitationWavelengthUnit',
10649                'Fluor',
10650                'IlluminationType',
10651                'NDFilter',
10652                'PinholeSize',
10653                'PinholeSizeUnit',
10654                'PockelCellSetting',
10655            )
10656            channels.append(
10657                f'<Channel ID="Channel:{index}:{c}" '
10658                f'SamplesPerPixel="{samples}"'
10659                f'{attributes}>'
10660                f'{lightpath}'
10661                '</Channel>'
10662            )
10663        channels = ''.join(channels)
10664
10665        # TODO: support more Image elements
10666        elements = OmeXml._elements(metadata, 'AcquisitionDate', 'Description')
10667
10668        name = OmeXml._attribute(metadata, 'Name', default=f'Image{index}')
10669        attributes = OmeXml._attributes(
10670            metadata,
10671            None,
10672            'SignificantBits',
10673            'PhysicalSizeX',
10674            'PhysicalSizeXUnit',
10675            'PhysicalSizeY',
10676            'PhysicalSizeYUnit',
10677            'PhysicalSizeZ',
10678            'PhysicalSizeZUnit',
10679            'TimeIncrement',
10680            'TimeIncrementUnit',
10681        )
10682        if separate > 1 or contig > 1:
10683            interleaved = 'false' if separate > 1 else 'true'
10684            interleaved = f' Interleaved="{interleaved}"'
10685        else:
10686            interleaved = ''
10687
10688        self.images.append(
10689            f'<Image ID="Image:{index}"{name}>'
10690            f'{elements}'
10691            f'<Pixels ID="Pixels:{index}" '
10692            f'DimensionOrder="{dimorder}" '
10693            f'Type="{dtype}"'
10694            f'{sizes}'
10695            f'{interleaved}'
10696            f'{attributes}>'
10697            f'{channels}'
10698            f'<TiffData IFD="{self.ifd}" PlaneCount="{planecount}"/>'
10699            f'{planes}'
10700            f'</Pixels>'
10701            f'{annotationref}'
10702            f'</Image>'
10703        )
10704        self.ifd += planecount
10705
10706    def tostring(self, declaration=False):
10707        """Return OME-XML string."""
10708        # TODO: support other top-level elements
10709        elements = ''.join(self.elements)
10710        images = ''.join(self.images)
10711        annotations = ''.join(self.annotations)
10712        if annotations:
10713            annotations = (
10714                f'<StructuredAnnotations>{annotations}</StructuredAnnotations>'
10715            )
10716        if declaration:
10717            declaration = '<?xml version="1.0" encoding="UTF-8"?>'
10718        else:
10719            declaration = ''
10720        xml = self.xml.format(
10721            declaration=declaration,
10722            images=images,
10723            annotations=annotations,
10724            elements=elements,
10725        )
10726        return xml
10727
10728    def __str__(self):
10729        """Return OME-XML string."""
10730        xml = self.tostring()
10731        try:
10732            from lxml import etree  # noqa: delayed import
10733
10734            parser = etree.XMLParser(remove_blank_text=True)
10735            tree = etree.fromstring(xml, parser)
10736            xml = etree.tostring(
10737                tree, encoding='utf-8', pretty_print=True, xml_declaration=True
10738            ).decode()
10739        except Exception as exc:
10740            warnings.warn(f'OmeXml.__str__: {exc}', UserWarning)
10741        except ImportError:
10742            pass
10743        return xml
10744
10745    @staticmethod
10746    def _escape(value):
10747        """Return escaped string of value."""
10748        if not isinstance(value, str):
10749            value = str(value)
10750        elif '&amp;' in value or '&gt;' in value or '&lt;' in value:
10751            return value
10752        value = value.replace('&', '&amp;')
10753        value = value.replace('>', '&gt;')
10754        value = value.replace('<', '&lt;')
10755        return value
10756
10757    @staticmethod
10758    def _element(metadata, name, default=None):
10759        """Return XML formatted element if name in metadata."""
10760        value = metadata.get(name, default)
10761        if value is None:
10762            return None
10763        return f'<{name}>{OmeXml._escape(value)}</{name}>'
10764
10765    @staticmethod
10766    def _elements(metadata, *names):
10767        """Return XML formatted elements."""
10768        if not metadata:
10769            return ''
10770        elements = (OmeXml._element(metadata, name) for name in names)
10771        return ''.join(e for e in elements if e)
10772
10773    @staticmethod
10774    def _attribute(metadata, name, index=None, default=None):
10775        """Return XML formatted attribute if name in metadata."""
10776        value = metadata.get(name, default)
10777        if value is None:
10778            return None
10779        if index is not None:
10780            if isinstance(value, (list, tuple)):
10781                value = value[index]
10782            elif index > 0:
10783                raise TypeError(
10784                    f'{type(value).__name__!r} is not a list or tuple'
10785                )
10786        return f' {name}="{OmeXml._escape(value)}"'
10787
10788    @staticmethod
10789    def _attributes(metadata, index_, *names):
10790        """Return XML formatted attributes."""
10791        if not metadata:
10792            return ''
10793        if index_ is None:
10794            attributes = (OmeXml._attribute(metadata, name) for name in names)
10795        elif isinstance(metadata, (list, tuple)):
10796            metadata = metadata[index_]
10797            attributes = (OmeXml._attribute(metadata, name) for name in names)
10798        elif isinstance(metadata, dict):
10799            attributes = (
10800                OmeXml._attribute(metadata, name, index_) for name in names
10801            )
10802        return ''.join(a for a in attributes if a)
10803
10804    @staticmethod
10805    def _reference(metadata, name):
10806        """Return XML formatted reference element."""
10807        value = metadata.get(name, None)
10808        if value is None:
10809            return ''
10810        try:
10811            value = value['ID']
10812        except KeyError:
10813            pass
10814        return f'<{name} ID="{OmeXml._escape(value)}"/>'
10815
10816    @staticmethod
10817    def validate(omexml, omexsd=None, assert_=True, _schema=[]):
10818        """Return if OME-XML is valid according to XMLSchema.
10819
10820        If 'assert_' is True, raise an AssertionError if validation fails.
10821
10822        On first run, this function takes several seconds to download and
10823        parse the 2016-06 OME XMLSchema.
10824
10825        """
10826        from lxml import etree  # noqa: delay import
10827
10828        if not _schema:
10829            if omexsd is None:
10830                omexsd = os.path.join(os.path.dirname(__file__), 'ome.xsd')
10831                if os.path.exists(omexsd):
10832                    with open(omexsd, 'rb') as fh:
10833                        omexsd = fh.read()
10834                else:
10835                    import urllib.request  # noqa: delay import
10836
10837                    with urllib.request.urlopen(
10838                        'https://www.openmicroscopy.org/'
10839                        'Schemas/OME/2016-06/ome.xsd'
10840                    ) as fh:
10841                        omexsd = fh.read()
10842            if omexsd.startswith(b'<?xml'):
10843                omexsd = omexsd.split(b'>', 1)[-1]
10844            try:
10845                _schema.append(
10846                    etree.XMLSchema(etree.fromstring(omexsd.decode()))
10847                )
10848            except Exception:
10849                # raise
10850                _schema.append(None)
10851        if _schema and _schema[0] is not None:
10852            if omexml.startswith('<?xml'):
10853                omexml = omexml.split('>', 1)[-1]
10854            tree = etree.fromstring(omexml)
10855            if assert_:
10856                _schema[0].assert_(tree)
10857                return True
10858            return _schema[0].validate(tree)
10859        return None
10860
10861
10862class LazyConst:
10863    """Class whose attributes are computed on first access from its methods."""
10864
10865    def __init__(self, cls):
10866        self._cls = cls
10867        self.__doc__ = cls.__doc__
10868        self.__module__ = cls.__module__
10869        self.__name__ = cls.__name__
10870        self.__qualname__ = cls.__qualname__
10871        self.lock = threading.RLock()
10872
10873    def __reduce__(self):
10874        # decorated class will be pickled by name
10875        return self._cls.__qualname__
10876
10877    def __getattr__(self, name):
10878        with self.lock:
10879            if name in self.__dict__:
10880                # another thread set attribute while awaiting lock
10881                return self.__dict__[name]
10882            func = getattr(self._cls, name)
10883            value = func() if callable(func) else func
10884            for attr in (
10885                '__module__',
10886                '__name__',
10887                '__qualname__',
10888                # '__doc__',
10889                # '__annotations__'
10890            ):
10891                try:
10892                    setattr(value, attr, getattr(func, attr))
10893                except AttributeError:
10894                    pass
10895            setattr(self, name, value)
10896        return value
10897
10898
10899@LazyConst
10900class TIFF:
10901    """Namespace for module constants."""
10902
10903    def CLASSIC_LE():
10904        class ClassicTiffLe:
10905            __slots__ = ()
10906            version = 42
10907            byteorder = '<'
10908            offsetsize = 4
10909            offsetformat = '<I'
10910            tagnosize = 2
10911            tagnoformat = '<H'
10912            tagsize = 12
10913            tagformat1 = '<HH'
10914            tagformat2 = '<I4s'
10915            tagoffsetthreshold = 4
10916
10917        return ClassicTiffLe
10918
10919    def CLASSIC_BE():
10920        class ClassicTiffBe:
10921            __slots__ = ()
10922            version = 42
10923            byteorder = '>'
10924            offsetsize = 4
10925            offsetformat = '>I'
10926            tagnosize = 2
10927            tagnoformat = '>H'
10928            tagsize = 12
10929            tagformat1 = '>HH'
10930            tagformat2 = '>I4s'
10931            tagoffsetthreshold = 4
10932
10933        return ClassicTiffBe
10934
10935    def BIG_LE():
10936        class BigTiffLe:
10937            __slots__ = ()
10938            version = 43
10939            byteorder = '<'
10940            offsetsize = 8
10941            offsetformat = '<Q'
10942            tagnosize = 8
10943            tagnoformat = '<Q'
10944            tagsize = 20
10945            tagformat1 = '<HH'
10946            tagformat2 = '<Q8s'
10947            tagoffsetthreshold = 8
10948
10949        return BigTiffLe
10950
10951    def BIG_BE():
10952        class BigTiffBe:
10953            __slots__ = ()
10954            version = 43
10955            byteorder = '>'
10956            offsetsize = 8
10957            offsetformat = '>Q'
10958            tagnosize = 8
10959            tagnoformat = '>Q'
10960            tagsize = 20
10961            tagformat1 = '>HH'
10962            tagformat2 = '>Q8s'
10963            tagoffsetthreshold = 8
10964
10965        return BigTiffBe
10966
10967    def NDPI_LE():
10968        class NdpiTiffLe:
10969            __slots__ = ()
10970            version = 42
10971            byteorder = '<'
10972            offsetsize = 8  # NDPI uses 8 bytes IFD and tag offsets
10973            offsetformat = '<Q'
10974            tagnosize = 2
10975            tagnoformat = '<H'
10976            tagsize = 12  # 16 after patching
10977            tagformat1 = '<HH'
10978            tagformat2 = '<I8s'  # after patching
10979            tagoffsetthreshold = 4
10980
10981        return NdpiTiffLe
10982
10983    def TAGS():
10984        # TIFF tag codes and names from TIFF6, TIFF/EP, EXIF, and other specs
10985        # TODO: divide into baseline, exif, private, ... tags
10986        return TiffTagRegistry(
10987            (
10988                (11, 'ProcessingSoftware'),
10989                (254, 'NewSubfileType'),
10990                (255, 'SubfileType'),
10991                (256, 'ImageWidth'),
10992                (257, 'ImageLength'),
10993                (258, 'BitsPerSample'),
10994                (259, 'Compression'),
10995                (262, 'PhotometricInterpretation'),
10996                (263, 'Thresholding'),
10997                (264, 'CellWidth'),
10998                (265, 'CellLength'),
10999                (266, 'FillOrder'),
11000                (269, 'DocumentName'),
11001                (270, 'ImageDescription'),
11002                (271, 'Make'),
11003                (272, 'Model'),
11004                (273, 'StripOffsets'),
11005                (274, 'Orientation'),
11006                (277, 'SamplesPerPixel'),
11007                (278, 'RowsPerStrip'),
11008                (279, 'StripByteCounts'),
11009                (280, 'MinSampleValue'),
11010                (281, 'MaxSampleValue'),
11011                (282, 'XResolution'),
11012                (283, 'YResolution'),
11013                (284, 'PlanarConfiguration'),
11014                (285, 'PageName'),
11015                (286, 'XPosition'),
11016                (287, 'YPosition'),
11017                (288, 'FreeOffsets'),
11018                (289, 'FreeByteCounts'),
11019                (290, 'GrayResponseUnit'),
11020                (291, 'GrayResponseCurve'),
11021                (292, 'T4Options'),
11022                (293, 'T6Options'),
11023                (296, 'ResolutionUnit'),
11024                (297, 'PageNumber'),
11025                (300, 'ColorResponseUnit'),
11026                (301, 'TransferFunction'),
11027                (305, 'Software'),
11028                (306, 'DateTime'),
11029                (315, 'Artist'),
11030                (316, 'HostComputer'),
11031                (317, 'Predictor'),
11032                (318, 'WhitePoint'),
11033                (319, 'PrimaryChromaticities'),
11034                (320, 'ColorMap'),
11035                (321, 'HalftoneHints'),
11036                (322, 'TileWidth'),
11037                (323, 'TileLength'),
11038                (324, 'TileOffsets'),
11039                (325, 'TileByteCounts'),
11040                (326, 'BadFaxLines'),
11041                (327, 'CleanFaxData'),
11042                (328, 'ConsecutiveBadFaxLines'),
11043                (330, 'SubIFDs'),
11044                (332, 'InkSet'),
11045                (333, 'InkNames'),
11046                (334, 'NumberOfInks'),
11047                (336, 'DotRange'),
11048                (337, 'TargetPrinter'),
11049                (338, 'ExtraSamples'),
11050                (339, 'SampleFormat'),
11051                (340, 'SMinSampleValue'),
11052                (341, 'SMaxSampleValue'),
11053                (342, 'TransferRange'),
11054                (343, 'ClipPath'),
11055                (344, 'XClipPathUnits'),
11056                (345, 'YClipPathUnits'),
11057                (346, 'Indexed'),
11058                (347, 'JPEGTables'),
11059                (351, 'OPIProxy'),
11060                (400, 'GlobalParametersIFD'),
11061                (401, 'ProfileType'),
11062                (402, 'FaxProfile'),
11063                (403, 'CodingMethods'),
11064                (404, 'VersionYear'),
11065                (405, 'ModeNumber'),
11066                (433, 'Decode'),
11067                (434, 'DefaultImageColor'),
11068                (435, 'T82Options'),
11069                (437, 'JPEGTables'),  # 347
11070                (512, 'JPEGProc'),
11071                (513, 'JPEGInterchangeFormat'),
11072                (514, 'JPEGInterchangeFormatLength'),
11073                (515, 'JPEGRestartInterval'),
11074                (517, 'JPEGLosslessPredictors'),
11075                (518, 'JPEGPointTransforms'),
11076                (519, 'JPEGQTables'),
11077                (520, 'JPEGDCTables'),
11078                (521, 'JPEGACTables'),
11079                (529, 'YCbCrCoefficients'),
11080                (530, 'YCbCrSubSampling'),
11081                (531, 'YCbCrPositioning'),
11082                (532, 'ReferenceBlackWhite'),
11083                (559, 'StripRowCounts'),
11084                (700, 'XMP'),  # XMLPacket
11085                (769, 'GDIGamma'),  # GDI+
11086                (770, 'ICCProfileDescriptor'),  # GDI+
11087                (771, 'SRGBRenderingIntent'),  # GDI+
11088                (800, 'ImageTitle'),  # GDI+
11089                (999, 'USPTO_Miscellaneous'),
11090                (4864, 'AndorId'),  # TODO, Andor Technology 4864 - 5030
11091                (4869, 'AndorTemperature'),
11092                (4876, 'AndorExposureTime'),
11093                (4878, 'AndorKineticCycleTime'),
11094                (4879, 'AndorAccumulations'),
11095                (4881, 'AndorAcquisitionCycleTime'),
11096                (4882, 'AndorReadoutTime'),
11097                (4884, 'AndorPhotonCounting'),
11098                (4885, 'AndorEmDacLevel'),
11099                (4890, 'AndorFrames'),
11100                (4896, 'AndorHorizontalFlip'),
11101                (4897, 'AndorVerticalFlip'),
11102                (4898, 'AndorClockwise'),
11103                (4899, 'AndorCounterClockwise'),
11104                (4904, 'AndorVerticalClockVoltage'),
11105                (4905, 'AndorVerticalShiftSpeed'),
11106                (4907, 'AndorPreAmpSetting'),
11107                (4908, 'AndorCameraSerial'),
11108                (4911, 'AndorActualTemperature'),
11109                (4912, 'AndorBaselineClamp'),
11110                (4913, 'AndorPrescans'),
11111                (4914, 'AndorModel'),
11112                (4915, 'AndorChipSizeX'),
11113                (4916, 'AndorChipSizeY'),
11114                (4944, 'AndorBaselineOffset'),
11115                (4966, 'AndorSoftwareVersion'),
11116                (18246, 'Rating'),
11117                (18247, 'XP_DIP_XML'),
11118                (18248, 'StitchInfo'),
11119                (18249, 'RatingPercent'),
11120                (20481, 'ResolutionXUnit'),  # GDI+
11121                (20482, 'ResolutionYUnit'),  # GDI+
11122                (20483, 'ResolutionXLengthUnit'),  # GDI+
11123                (20484, 'ResolutionYLengthUnit'),  # GDI+
11124                (20485, 'PrintFlags'),  # GDI+
11125                (20486, 'PrintFlagsVersion'),  # GDI+
11126                (20487, 'PrintFlagsCrop'),  # GDI+
11127                (20488, 'PrintFlagsBleedWidth'),  # GDI+
11128                (20489, 'PrintFlagsBleedWidthScale'),  # GDI+
11129                (20490, 'HalftoneLPI'),  # GDI+
11130                (20491, 'HalftoneLPIUnit'),  # GDI+
11131                (20492, 'HalftoneDegree'),  # GDI+
11132                (20493, 'HalftoneShape'),  # GDI+
11133                (20494, 'HalftoneMisc'),  # GDI+
11134                (20495, 'HalftoneScreen'),  # GDI+
11135                (20496, 'JPEGQuality'),  # GDI+
11136                (20497, 'GridSize'),  # GDI+
11137                (20498, 'ThumbnailFormat'),  # GDI+
11138                (20499, 'ThumbnailWidth'),  # GDI+
11139                (20500, 'ThumbnailHeight'),  # GDI+
11140                (20501, 'ThumbnailColorDepth'),  # GDI+
11141                (20502, 'ThumbnailPlanes'),  # GDI+
11142                (20503, 'ThumbnailRawBytes'),  # GDI+
11143                (20504, 'ThumbnailSize'),  # GDI+
11144                (20505, 'ThumbnailCompressedSize'),  # GDI+
11145                (20506, 'ColorTransferFunction'),  # GDI+
11146                (20507, 'ThumbnailData'),
11147                (20512, 'ThumbnailImageWidth'),  # GDI+
11148                (20513, 'ThumbnailImageHeight'),  # GDI+
11149                (20514, 'ThumbnailBitsPerSample'),  # GDI+
11150                (20515, 'ThumbnailCompression'),
11151                (20516, 'ThumbnailPhotometricInterp'),  # GDI+
11152                (20517, 'ThumbnailImageDescription'),  # GDI+
11153                (20518, 'ThumbnailEquipMake'),  # GDI+
11154                (20519, 'ThumbnailEquipModel'),  # GDI+
11155                (20520, 'ThumbnailStripOffsets'),  # GDI+
11156                (20521, 'ThumbnailOrientation'),  # GDI+
11157                (20522, 'ThumbnailSamplesPerPixel'),  # GDI+
11158                (20523, 'ThumbnailRowsPerStrip'),  # GDI+
11159                (20524, 'ThumbnailStripBytesCount'),  # GDI+
11160                (20525, 'ThumbnailResolutionX'),
11161                (20526, 'ThumbnailResolutionY'),
11162                (20527, 'ThumbnailPlanarConfig'),  # GDI+
11163                (20528, 'ThumbnailResolutionUnit'),
11164                (20529, 'ThumbnailTransferFunction'),
11165                (20530, 'ThumbnailSoftwareUsed'),  # GDI+
11166                (20531, 'ThumbnailDateTime'),  # GDI+
11167                (20532, 'ThumbnailArtist'),  # GDI+
11168                (20533, 'ThumbnailWhitePoint'),  # GDI+
11169                (20534, 'ThumbnailPrimaryChromaticities'),  # GDI+
11170                (20535, 'ThumbnailYCbCrCoefficients'),  # GDI+
11171                (20536, 'ThumbnailYCbCrSubsampling'),  # GDI+
11172                (20537, 'ThumbnailYCbCrPositioning'),
11173                (20538, 'ThumbnailRefBlackWhite'),  # GDI+
11174                (20539, 'ThumbnailCopyRight'),  # GDI+
11175                (20545, 'InteroperabilityIndex'),
11176                (20546, 'InteroperabilityVersion'),
11177                (20624, 'LuminanceTable'),
11178                (20625, 'ChrominanceTable'),
11179                (20736, 'FrameDelay'),  # GDI+
11180                (20737, 'LoopCount'),  # GDI+
11181                (20738, 'GlobalPalette'),  # GDI+
11182                (20739, 'IndexBackground'),  # GDI+
11183                (20740, 'IndexTransparent'),  # GDI+
11184                (20752, 'PixelUnit'),  # GDI+
11185                (20753, 'PixelPerUnitX'),  # GDI+
11186                (20754, 'PixelPerUnitY'),  # GDI+
11187                (20755, 'PaletteHistogram'),  # GDI+
11188                (28672, 'SonyRawFileType'),  # Sony ARW
11189                (28722, 'VignettingCorrParams'),  # Sony ARW
11190                (28725, 'ChromaticAberrationCorrParams'),  # Sony ARW
11191                (28727, 'DistortionCorrParams'),  # Sony ARW
11192                # Private tags >= 32768
11193                (32781, 'ImageID'),
11194                (32931, 'WangTag1'),
11195                (32932, 'WangAnnotation'),
11196                (32933, 'WangTag3'),
11197                (32934, 'WangTag4'),
11198                (32953, 'ImageReferencePoints'),
11199                (32954, 'RegionXformTackPoint'),
11200                (32955, 'WarpQuadrilateral'),
11201                (32956, 'AffineTransformMat'),
11202                (32995, 'Matteing'),
11203                (32996, 'DataType'),  # use SampleFormat
11204                (32997, 'ImageDepth'),
11205                (32998, 'TileDepth'),
11206                (33300, 'ImageFullWidth'),
11207                (33301, 'ImageFullLength'),
11208                (33302, 'TextureFormat'),
11209                (33303, 'TextureWrapModes'),
11210                (33304, 'FieldOfViewCotangent'),
11211                (33305, 'MatrixWorldToScreen'),
11212                (33306, 'MatrixWorldToCamera'),
11213                (33405, 'Model2'),
11214                (33421, 'CFARepeatPatternDim'),
11215                (33422, 'CFAPattern'),
11216                (33423, 'BatteryLevel'),
11217                (33424, 'KodakIFD'),
11218                (33434, 'ExposureTime'),
11219                (33437, 'FNumber'),
11220                (33432, 'Copyright'),
11221                (33445, 'MDFileTag'),
11222                (33446, 'MDScalePixel'),
11223                (33447, 'MDColorTable'),
11224                (33448, 'MDLabName'),
11225                (33449, 'MDSampleInfo'),
11226                (33450, 'MDPrepDate'),
11227                (33451, 'MDPrepTime'),
11228                (33452, 'MDFileUnits'),
11229                (33471, 'OlympusINI'),
11230                (33550, 'ModelPixelScaleTag'),
11231                (33560, 'OlympusSIS'),  # see also 33471 and 34853
11232                (33589, 'AdventScale'),
11233                (33590, 'AdventRevision'),
11234                (33628, 'UIC1tag'),  # Metamorph  Universal Imaging Corp STK
11235                (33629, 'UIC2tag'),
11236                (33630, 'UIC3tag'),
11237                (33631, 'UIC4tag'),
11238                (33723, 'IPTCNAA'),
11239                (33858, 'ExtendedTagsOffset'),  # DEFF points IFD with tags
11240                (33918, 'IntergraphPacketData'),  # INGRPacketDataTag
11241                (33919, 'IntergraphFlagRegisters'),  # INGRFlagRegisters
11242                (33920, 'IntergraphMatrixTag'),  # IrasBTransformationMatrix
11243                (33921, 'INGRReserved'),
11244                (33922, 'ModelTiepointTag'),
11245                (33923, 'LeicaMagic'),
11246                (34016, 'Site'),  # 34016..34032 ANSI IT8 TIFF/IT
11247                (34017, 'ColorSequence'),
11248                (34018, 'IT8Header'),
11249                (34019, 'RasterPadding'),
11250                (34020, 'BitsPerRunLength'),
11251                (34021, 'BitsPerExtendedRunLength'),
11252                (34022, 'ColorTable'),
11253                (34023, 'ImageColorIndicator'),
11254                (34024, 'BackgroundColorIndicator'),
11255                (34025, 'ImageColorValue'),
11256                (34026, 'BackgroundColorValue'),
11257                (34027, 'PixelIntensityRange'),
11258                (34028, 'TransparencyIndicator'),
11259                (34029, 'ColorCharacterization'),
11260                (34030, 'HCUsage'),
11261                (34031, 'TrapIndicator'),
11262                (34032, 'CMYKEquivalent'),
11263                (34118, 'CZ_SEM'),  # Zeiss SEM
11264                (34152, 'AFCP_IPTC'),
11265                (34232, 'PixelMagicJBIGOptions'),  # EXIF, also TI FrameCount
11266                (34263, 'JPLCartoIFD'),
11267                (34122, 'IPLAB'),  # number of images
11268                (34264, 'ModelTransformationTag'),
11269                (34306, 'WB_GRGBLevels'),  # Leaf MOS
11270                (34310, 'LeafData'),
11271                (34361, 'MM_Header'),
11272                (34362, 'MM_Stamp'),
11273                (34363, 'MM_Unknown'),
11274                (34377, 'ImageResources'),  # Photoshop
11275                (34386, 'MM_UserBlock'),
11276                (34412, 'CZ_LSMINFO'),
11277                (34665, 'ExifTag'),
11278                (34675, 'InterColorProfile'),  # ICCProfile
11279                (34680, 'FEI_SFEG'),  #
11280                (34682, 'FEI_HELIOS'),  #
11281                (34683, 'FEI_TITAN'),  #
11282                (34687, 'FXExtensions'),
11283                (34688, 'MultiProfiles'),
11284                (34689, 'SharedData'),
11285                (34690, 'T88Options'),
11286                (34710, 'MarCCD'),  # offset to MarCCD header
11287                (34732, 'ImageLayer'),
11288                (34735, 'GeoKeyDirectoryTag'),
11289                (34736, 'GeoDoubleParamsTag'),
11290                (34737, 'GeoAsciiParamsTag'),
11291                (34750, 'JBIGOptions'),
11292                (34821, 'PIXTIFF'),  # ? Pixel Translations Inc
11293                (34850, 'ExposureProgram'),
11294                (34852, 'SpectralSensitivity'),
11295                (34853, 'GPSTag'),  # GPSIFD  also OlympusSIS2
11296                (34853, 'OlympusSIS2'),
11297                (34855, 'ISOSpeedRatings'),
11298                (34855, 'PhotographicSensitivity'),
11299                (34856, 'OECF'),  # optoelectric conversion factor
11300                (34857, 'Interlace'),
11301                (34858, 'TimeZoneOffset'),
11302                (34859, 'SelfTimerMode'),
11303                (34864, 'SensitivityType'),
11304                (34865, 'StandardOutputSensitivity'),
11305                (34866, 'RecommendedExposureIndex'),
11306                (34867, 'ISOSpeed'),
11307                (34868, 'ISOSpeedLatitudeyyy'),
11308                (34869, 'ISOSpeedLatitudezzz'),
11309                (34908, 'HylaFAXFaxRecvParams'),
11310                (34909, 'HylaFAXFaxSubAddress'),
11311                (34910, 'HylaFAXFaxRecvTime'),
11312                (34911, 'FaxDcs'),
11313                (34929, 'FedexEDR'),
11314                (34954, 'LeafSubIFD'),
11315                (34959, 'Aphelion1'),
11316                (34960, 'Aphelion2'),
11317                (34961, 'AphelionInternal'),  # ADCIS
11318                (36864, 'ExifVersion'),
11319                (36867, 'DateTimeOriginal'),
11320                (36868, 'DateTimeDigitized'),
11321                (36873, 'GooglePlusUploadCode'),
11322                (36880, 'OffsetTime'),
11323                (36881, 'OffsetTimeOriginal'),
11324                (36882, 'OffsetTimeDigitized'),
11325                # TODO, Pilatus/CHESS/TV6 36864..37120 conflicting with Exif
11326                (36864, 'TVX_Unknown'),
11327                (36865, 'TVX_NumExposure'),
11328                (36866, 'TVX_NumBackground'),
11329                (36867, 'TVX_ExposureTime'),
11330                (36868, 'TVX_BackgroundTime'),
11331                (36870, 'TVX_Unknown'),
11332                (36873, 'TVX_SubBpp'),
11333                (36874, 'TVX_SubWide'),
11334                (36875, 'TVX_SubHigh'),
11335                (36876, 'TVX_BlackLevel'),
11336                (36877, 'TVX_DarkCurrent'),
11337                (36878, 'TVX_ReadNoise'),
11338                (36879, 'TVX_DarkCurrentNoise'),
11339                (36880, 'TVX_BeamMonitor'),
11340                (37120, 'TVX_UserVariables'),  # A/D values
11341                (37121, 'ComponentsConfiguration'),
11342                (37122, 'CompressedBitsPerPixel'),
11343                (37377, 'ShutterSpeedValue'),
11344                (37378, 'ApertureValue'),
11345                (37379, 'BrightnessValue'),
11346                (37380, 'ExposureBiasValue'),
11347                (37381, 'MaxApertureValue'),
11348                (37382, 'SubjectDistance'),
11349                (37383, 'MeteringMode'),
11350                (37384, 'LightSource'),
11351                (37385, 'Flash'),
11352                (37386, 'FocalLength'),
11353                (37387, 'FlashEnergy'),  # 37387
11354                (37388, 'SpatialFrequencyResponse'),  # 37388
11355                (37389, 'Noise'),
11356                (37390, 'FocalPlaneXResolution'),
11357                (37391, 'FocalPlaneYResolution'),
11358                (37392, 'FocalPlaneResolutionUnit'),
11359                (37393, 'ImageNumber'),
11360                (37394, 'SecurityClassification'),
11361                (37395, 'ImageHistory'),
11362                (37396, 'SubjectLocation'),
11363                (37397, 'ExposureIndex'),
11364                (37398, 'TIFFEPStandardID'),
11365                (37399, 'SensingMethod'),
11366                (37434, 'CIP3DataFile'),
11367                (37435, 'CIP3Sheet'),
11368                (37436, 'CIP3Side'),
11369                (37439, 'StoNits'),
11370                (37500, 'MakerNote'),
11371                (37510, 'UserComment'),
11372                (37520, 'SubsecTime'),
11373                (37521, 'SubsecTimeOriginal'),
11374                (37522, 'SubsecTimeDigitized'),
11375                (37679, 'MODIText'),  # Microsoft Office Document Imaging
11376                (37680, 'MODIOLEPropertySetStorage'),
11377                (37681, 'MODIPositioning'),
11378                (37706, 'TVIPS'),  # offset to TemData structure
11379                (37707, 'TVIPS1'),
11380                (37708, 'TVIPS2'),  # same TemData structure as undefined
11381                (37724, 'ImageSourceData'),  # Photoshop
11382                (37888, 'Temperature'),
11383                (37889, 'Humidity'),
11384                (37890, 'Pressure'),
11385                (37891, 'WaterDepth'),
11386                (37892, 'Acceleration'),
11387                (37893, 'CameraElevationAngle'),
11388                (40000, 'XPos'),  # Janelia
11389                (40001, 'YPos'),
11390                (40002, 'ZPos'),
11391                (40001, 'MC_IpWinScal'),  # Media Cybernetics
11392                (40001, 'RecipName'),  # MS FAX
11393                (40002, 'RecipNumber'),
11394                (40003, 'SenderName'),
11395                (40004, 'Routing'),
11396                (40005, 'CallerId'),
11397                (40006, 'TSID'),
11398                (40007, 'CSID'),
11399                (40008, 'FaxTime'),
11400                (40100, 'MC_IdOld'),
11401                (40106, 'MC_Unknown'),
11402                (40965, 'InteroperabilityTag'),  # InteropOffset
11403                (40091, 'XPTitle'),
11404                (40092, 'XPComment'),
11405                (40093, 'XPAuthor'),
11406                (40094, 'XPKeywords'),
11407                (40095, 'XPSubject'),
11408                (40960, 'FlashpixVersion'),
11409                (40961, 'ColorSpace'),
11410                (40962, 'PixelXDimension'),
11411                (40963, 'PixelYDimension'),
11412                (40964, 'RelatedSoundFile'),
11413                (40976, 'SamsungRawPointersOffset'),
11414                (40977, 'SamsungRawPointersLength'),
11415                (41217, 'SamsungRawByteOrder'),
11416                (41218, 'SamsungRawUnknown'),
11417                (41483, 'FlashEnergy'),
11418                (41484, 'SpatialFrequencyResponse'),
11419                (41485, 'Noise'),  # 37389
11420                (41486, 'FocalPlaneXResolution'),  # 37390
11421                (41487, 'FocalPlaneYResolution'),  # 37391
11422                (41488, 'FocalPlaneResolutionUnit'),  # 37392
11423                (41489, 'ImageNumber'),  # 37393
11424                (41490, 'SecurityClassification'),  # 37394
11425                (41491, 'ImageHistory'),  # 37395
11426                (41492, 'SubjectLocation'),  # 37395
11427                (41493, 'ExposureIndex '),  # 37397
11428                (41494, 'TIFF-EPStandardID'),
11429                (41495, 'SensingMethod'),  # 37399
11430                (41728, 'FileSource'),
11431                (41729, 'SceneType'),
11432                (41730, 'CFAPattern'),  # 33422
11433                (41985, 'CustomRendered'),
11434                (41986, 'ExposureMode'),
11435                (41987, 'WhiteBalance'),
11436                (41988, 'DigitalZoomRatio'),
11437                (41989, 'FocalLengthIn35mmFilm'),
11438                (41990, 'SceneCaptureType'),
11439                (41991, 'GainControl'),
11440                (41992, 'Contrast'),
11441                (41993, 'Saturation'),
11442                (41994, 'Sharpness'),
11443                (41995, 'DeviceSettingDescription'),
11444                (41996, 'SubjectDistanceRange'),
11445                (42016, 'ImageUniqueID'),
11446                (42032, 'CameraOwnerName'),
11447                (42033, 'BodySerialNumber'),
11448                (42034, 'LensSpecification'),
11449                (42035, 'LensMake'),
11450                (42036, 'LensModel'),
11451                (42037, 'LensSerialNumber'),
11452                (42080, 'CompositeImage'),
11453                (42081, 'SourceImageNumberCompositeImage'),
11454                (42082, 'SourceExposureTimesCompositeImage'),
11455                (42112, 'GDAL_METADATA'),
11456                (42113, 'GDAL_NODATA'),
11457                (42240, 'Gamma'),
11458                (43314, 'NIHImageHeader'),
11459                (44992, 'ExpandSoftware'),
11460                (44993, 'ExpandLens'),
11461                (44994, 'ExpandFilm'),
11462                (44995, 'ExpandFilterLens'),
11463                (44996, 'ExpandScanner'),
11464                (44997, 'ExpandFlashLamp'),
11465                (48129, 'PixelFormat'),  # HDP and WDP
11466                (48130, 'Transformation'),
11467                (48131, 'Uncompressed'),
11468                (48132, 'ImageType'),
11469                (48256, 'ImageWidth'),  # 256
11470                (48257, 'ImageHeight'),
11471                (48258, 'WidthResolution'),
11472                (48259, 'HeightResolution'),
11473                (48320, 'ImageOffset'),
11474                (48321, 'ImageByteCount'),
11475                (48322, 'AlphaOffset'),
11476                (48323, 'AlphaByteCount'),
11477                (48324, 'ImageDataDiscard'),
11478                (48325, 'AlphaDataDiscard'),
11479                (50003, 'KodakAPP3'),
11480                (50215, 'OceScanjobDescription'),
11481                (50216, 'OceApplicationSelector'),
11482                (50217, 'OceIdentificationNumber'),
11483                (50218, 'OceImageLogicCharacteristics'),
11484                (50255, 'Annotations'),
11485                (50288, 'MC_Id'),  # Media Cybernetics
11486                (50289, 'MC_XYPosition'),
11487                (50290, 'MC_ZPosition'),
11488                (50291, 'MC_XYCalibration'),
11489                (50292, 'MC_LensCharacteristics'),
11490                (50293, 'MC_ChannelName'),
11491                (50294, 'MC_ExcitationWavelength'),
11492                (50295, 'MC_TimeStamp'),
11493                (50296, 'MC_FrameProperties'),
11494                (50341, 'PrintImageMatching'),
11495                (50495, 'PCO_RAW'),  # TODO, PCO CamWare
11496                (50547, 'OriginalFileName'),
11497                (50560, 'USPTO_OriginalContentType'),  # US Patent Office
11498                (50561, 'USPTO_RotationCode'),
11499                (50648, 'CR2Unknown1'),
11500                (50649, 'CR2Unknown2'),
11501                (50656, 'CR2CFAPattern'),
11502                (50674, 'LercParameters'),  # ESGI 50674 .. 50677
11503                (50706, 'DNGVersion'),  # DNG 50706 .. 51114
11504                (50707, 'DNGBackwardVersion'),
11505                (50708, 'UniqueCameraModel'),
11506                (50709, 'LocalizedCameraModel'),
11507                (50710, 'CFAPlaneColor'),
11508                (50711, 'CFALayout'),
11509                (50712, 'LinearizationTable'),
11510                (50713, 'BlackLevelRepeatDim'),
11511                (50714, 'BlackLevel'),
11512                (50715, 'BlackLevelDeltaH'),
11513                (50716, 'BlackLevelDeltaV'),
11514                (50717, 'WhiteLevel'),
11515                (50718, 'DefaultScale'),
11516                (50719, 'DefaultCropOrigin'),
11517                (50720, 'DefaultCropSize'),
11518                (50721, 'ColorMatrix1'),
11519                (50722, 'ColorMatrix2'),
11520                (50723, 'CameraCalibration1'),
11521                (50724, 'CameraCalibration2'),
11522                (50725, 'ReductionMatrix1'),
11523                (50726, 'ReductionMatrix2'),
11524                (50727, 'AnalogBalance'),
11525                (50728, 'AsShotNeutral'),
11526                (50729, 'AsShotWhiteXY'),
11527                (50730, 'BaselineExposure'),
11528                (50731, 'BaselineNoise'),
11529                (50732, 'BaselineSharpness'),
11530                (50733, 'BayerGreenSplit'),
11531                (50734, 'LinearResponseLimit'),
11532                (50735, 'CameraSerialNumber'),
11533                (50736, 'LensInfo'),
11534                (50737, 'ChromaBlurRadius'),
11535                (50738, 'AntiAliasStrength'),
11536                (50739, 'ShadowScale'),
11537                (50740, 'DNGPrivateData'),
11538                (50741, 'MakerNoteSafety'),
11539                (50752, 'RawImageSegmentation'),
11540                (50778, 'CalibrationIlluminant1'),
11541                (50779, 'CalibrationIlluminant2'),
11542                (50780, 'BestQualityScale'),
11543                (50781, 'RawDataUniqueID'),
11544                (50784, 'AliasLayerMetadata'),
11545                (50827, 'OriginalRawFileName'),
11546                (50828, 'OriginalRawFileData'),
11547                (50829, 'ActiveArea'),
11548                (50830, 'MaskedAreas'),
11549                (50831, 'AsShotICCProfile'),
11550                (50832, 'AsShotPreProfileMatrix'),
11551                (50833, 'CurrentICCProfile'),
11552                (50834, 'CurrentPreProfileMatrix'),
11553                (50838, 'IJMetadataByteCounts'),
11554                (50839, 'IJMetadata'),
11555                (50844, 'RPCCoefficientTag'),
11556                (50879, 'ColorimetricReference'),
11557                (50885, 'SRawType'),
11558                (50898, 'PanasonicTitle'),
11559                (50899, 'PanasonicTitle2'),
11560                (50908, 'RSID'),  # DGIWG
11561                (50909, 'GEO_METADATA'),  # DGIWG XML
11562                (50931, 'CameraCalibrationSignature'),
11563                (50932, 'ProfileCalibrationSignature'),
11564                (50933, 'ProfileIFD'),  # EXTRACAMERAPROFILES
11565                (50934, 'AsShotProfileName'),
11566                (50935, 'NoiseReductionApplied'),
11567                (50936, 'ProfileName'),
11568                (50937, 'ProfileHueSatMapDims'),
11569                (50938, 'ProfileHueSatMapData1'),
11570                (50939, 'ProfileHueSatMapData2'),
11571                (50940, 'ProfileToneCurve'),
11572                (50941, 'ProfileEmbedPolicy'),
11573                (50942, 'ProfileCopyright'),
11574                (50964, 'ForwardMatrix1'),
11575                (50965, 'ForwardMatrix2'),
11576                (50966, 'PreviewApplicationName'),
11577                (50967, 'PreviewApplicationVersion'),
11578                (50968, 'PreviewSettingsName'),
11579                (50969, 'PreviewSettingsDigest'),
11580                (50970, 'PreviewColorSpace'),
11581                (50971, 'PreviewDateTime'),
11582                (50972, 'RawImageDigest'),
11583                (50973, 'OriginalRawFileDigest'),
11584                (50974, 'SubTileBlockSize'),
11585                (50975, 'RowInterleaveFactor'),
11586                (50981, 'ProfileLookTableDims'),
11587                (50982, 'ProfileLookTableData'),
11588                (51008, 'OpcodeList1'),
11589                (51009, 'OpcodeList2'),
11590                (51022, 'OpcodeList3'),
11591                (51023, 'FibicsXML'),  #
11592                (51041, 'NoiseProfile'),
11593                (51043, 'TimeCodes'),
11594                (51044, 'FrameRate'),
11595                (51058, 'TStop'),
11596                (51081, 'ReelName'),
11597                (51089, 'OriginalDefaultFinalSize'),
11598                (51090, 'OriginalBestQualitySize'),
11599                (51091, 'OriginalDefaultCropSize'),
11600                (51105, 'CameraLabel'),
11601                (51107, 'ProfileHueSatMapEncoding'),
11602                (51108, 'ProfileLookTableEncoding'),
11603                (51109, 'BaselineExposureOffset'),
11604                (51110, 'DefaultBlackRender'),
11605                (51111, 'NewRawImageDigest'),
11606                (51112, 'RawToPreviewGain'),
11607                (51113, 'CacheBlob'),
11608                (51114, 'CacheVersion'),
11609                (51123, 'MicroManagerMetadata'),
11610                (51125, 'DefaultUserCrop'),
11611                (51159, 'ZIFmetadata'),  # Objective Pathology Services
11612                (51160, 'ZIFannotations'),  # Objective Pathology Services
11613                (51177, 'DepthFormat'),
11614                (51178, 'DepthNear'),
11615                (51179, 'DepthFar'),
11616                (51180, 'DepthUnits'),
11617                (51181, 'DepthMeasureType'),
11618                (51182, 'EnhanceParams'),
11619                (59932, 'Padding'),
11620                (59933, 'OffsetSchema'),
11621                # Reusable Tags 65000-65535
11622                # (65000,  DimapDocumentXML'),
11623                # (65001, 'EER_XML'),
11624                # 65000-65112,  Photoshop Camera RAW EXIF tags
11625                # (65000, 'OwnerName'),
11626                # (65001, 'SerialNumber'),
11627                # (65002, 'Lens'),
11628                # (65024, 'KodakKDCPrivateIFD'),
11629                # (65100, 'RawFile'),
11630                # (65101, 'Converter'),
11631                # (65102, 'WhiteBalance'),
11632                # (65105, 'Exposure'),
11633                # (65106, 'Shadows'),
11634                # (65107, 'Brightness'),
11635                # (65108, 'Contrast'),
11636                # (65109, 'Saturation'),
11637                # (65110, 'Sharpness'),
11638                # (65111, 'Smoothness'),
11639                # (65112, 'MoireFilter'),
11640                (65200, 'FlexXML'),
11641            )
11642        )
11643
11644    def TAG_READERS():
11645        # map tag codes to import functions
11646        return {
11647            320: read_colormap,
11648            # 700: read_bytes,  # read_utf8,
11649            # 34377: read_bytes,
11650            33723: read_bytes,
11651            # 34675: read_bytes,
11652            33628: read_uic1tag,  # Universal Imaging Corp STK
11653            33629: read_uic2tag,
11654            33630: read_uic3tag,
11655            33631: read_uic4tag,
11656            34118: read_cz_sem,  # Carl Zeiss SEM
11657            34361: read_mm_header,  # Olympus FluoView
11658            34362: read_mm_stamp,
11659            34363: read_numpy,  # MM_Unknown
11660            34386: read_numpy,  # MM_UserBlock
11661            34412: read_cz_lsminfo,  # Carl Zeiss LSM
11662            34680: read_fei_metadata,  # S-FEG
11663            34682: read_fei_metadata,  # Helios NanoLab
11664            37706: read_tvips_header,  # TVIPS EMMENU
11665            37724: read_bytes,  # ImageSourceData
11666            33923: read_bytes,  # read_leica_magic
11667            43314: read_nih_image_header,
11668            # 40001: read_bytes,
11669            40100: read_bytes,
11670            50288: read_bytes,
11671            50296: read_bytes,
11672            50839: read_bytes,
11673            51123: read_json,
11674            33471: read_sis_ini,
11675            33560: read_sis,
11676            34665: read_exif_ifd,
11677            34853: read_gps_ifd,  # conflicts with OlympusSIS
11678            40965: read_interoperability_ifd,
11679            65426: read_numpy,  # NDPI McuStarts
11680            65432: read_numpy,  # NDPI McuStartsHighBytes
11681            65439: read_numpy,  # NDPI unknown
11682            65459: read_bytes,  # NDPI bytes, not string
11683        }
11684
11685    def TAG_TUPLE():
11686        # tags whose values must be stored as tuples
11687        return frozenset(
11688            (273, 279, 324, 325, 330, 338, 513, 514, 530, 531, 34736)
11689        )
11690
11691    def TAG_ATTRIBUTES():
11692        # map tag codes to TiffPage attribute names
11693        return {
11694            254: 'subfiletype',
11695            256: 'imagewidth',
11696            257: 'imagelength',
11697            # 258: 'bitspersample',  # set manually
11698            259: 'compression',
11699            262: 'photometric',
11700            266: 'fillorder',
11701            270: 'description',
11702            277: 'samplesperpixel',
11703            278: 'rowsperstrip',
11704            284: 'planarconfig',
11705            305: 'software',
11706            320: 'colormap',
11707            317: 'predictor',
11708            322: 'tilewidth',
11709            323: 'tilelength',
11710            330: 'subifds',
11711            338: 'extrasamples',
11712            # 339: 'sampleformat',  # set manually
11713            347: 'jpegtables',
11714            530: 'subsampling',
11715            32997: 'imagedepth',
11716            32998: 'tiledepth',
11717        }
11718
11719    def TAG_ENUM():
11720        # map tag codes to Enums
11721        class TAG_ENUM:
11722
11723            __slots__ = ('_codes',)
11724
11725            def __init__(self):
11726                self._codes = {
11727                    254: None,
11728                    255: None,
11729                    259: TIFF.COMPRESSION,
11730                    262: TIFF.PHOTOMETRIC,
11731                    263: None,
11732                    266: None,
11733                    274: None,
11734                    284: TIFF.PLANARCONFIG,
11735                    290: None,
11736                    296: None,
11737                    300: None,
11738                    317: None,
11739                    338: None,
11740                    339: TIFF.SAMPLEFORMAT,
11741                }
11742
11743            def __contains__(self, key):
11744                return key in self._codes
11745
11746            def __getitem__(self, key):
11747                value = self._codes[key]
11748                if value is not None:
11749                    return value
11750                if key == 254:
11751                    value = TIFF.FILETYPE
11752                elif key == 255:
11753                    value = TIFF.OFILETYPE
11754                elif key == 263:
11755                    value = TIFF.THRESHHOLD
11756                elif key == 266:
11757                    value = TIFF.FILLORDER
11758                elif key == 274:
11759                    value = TIFF.ORIENTATION
11760                elif key == 290:
11761                    value = TIFF.GRAYRESPONSEUNIT
11762                # elif key == 292:
11763                #     value = TIFF.GROUP3OPT
11764                # elif key == 293:
11765                #     value = TIFF.GROUP4OPT
11766                elif key == 296:
11767                    value = TIFF.RESUNIT
11768                elif key == 300:
11769                    value = TIFF.COLORRESPONSEUNIT
11770                elif key == 317:
11771                    value = TIFF.PREDICTOR
11772                elif key == 338:
11773                    value = TIFF.EXTRASAMPLE
11774                # elif key == 512:
11775                #     TIFF.JPEGPROC
11776                # elif key == 531:
11777                #     TIFF.YCBCRPOSITION
11778                else:
11779                    raise KeyError(key)
11780                self._codes[key] = value
11781                return value
11782
11783        return TAG_ENUM()
11784
11785    def FILETYPE():
11786        class FILETYPE(enum.IntFlag):
11787            UNDEFINED = 0
11788            REDUCEDIMAGE = 1
11789            PAGE = 2
11790            MASK = 4
11791            MACRO = 8  # Aperio SVS, or DNG Depth map
11792            ENHANCED = 16  # DNG
11793            DNG = 65536  # 65537: Alternative, 65540: Semantic mask
11794
11795        return FILETYPE
11796
11797    def OFILETYPE():
11798        class OFILETYPE(enum.IntEnum):
11799            UNDEFINED = 0
11800            IMAGE = 1
11801            REDUCEDIMAGE = 2
11802            PAGE = 3
11803
11804        return OFILETYPE
11805
11806    def COMPRESSION():
11807        class COMPRESSION(enum.IntEnum):
11808            NONE = 1  # Uncompressed
11809            CCITTRLE = 2  # CCITT 1D
11810            CCITT_T4 = 3  # T4/Group 3 Fax
11811            CCITT_T6 = 4  # T6/Group 4 Fax
11812            LZW = 5
11813            OJPEG = 6  # old-style JPEG
11814            JPEG = 7
11815            ADOBE_DEFLATE = 8
11816            JBIG_BW = 9
11817            JBIG_COLOR = 10
11818            JPEG_99 = 99
11819            KODAK_262 = 262
11820            JPEGXR_NDPI = 22610
11821            NEXT = 32766
11822            SONY_ARW = 32767
11823            PACKED_RAW = 32769
11824            SAMSUNG_SRW = 32770
11825            CCIRLEW = 32771
11826            SAMSUNG_SRW2 = 32772
11827            PACKBITS = 32773
11828            THUNDERSCAN = 32809
11829            IT8CTPAD = 32895
11830            IT8LW = 32896
11831            IT8MP = 32897
11832            IT8BL = 32898
11833            PIXARFILM = 32908
11834            PIXARLOG = 32909
11835            DEFLATE = 32946
11836            DCS = 32947
11837            APERIO_JP2000_YCBC = 33003  # Leica Aperio
11838            JPEG_2000_LOSSY = 33004  # BioFormats
11839            APERIO_JP2000_RGB = 33005  # Leica Aperio
11840            ALT_JPEG = 33007  # BioFormats
11841            JBIG = 34661
11842            SGILOG = 34676
11843            SGILOG24 = 34677
11844            JPEG2000 = 34712
11845            NIKON_NEF = 34713
11846            JBIG2 = 34715
11847            MDI_BINARY = 34718  # Microsoft Document Imaging
11848            MDI_PROGRESSIVE = 34719  # Microsoft Document Imaging
11849            MDI_VECTOR = 34720  # Microsoft Document Imaging
11850            LERC = 34887  # ESRI Lerc
11851            JPEG_LOSSY = 34892  # DNG
11852            LZMA = 34925
11853            ZSTD_DEPRECATED = 34926
11854            WEBP_DEPRECATED = 34927
11855            PNG = 34933  # Objective Pathology Services
11856            JPEGXR = 34934  # Objective Pathology Services
11857            ZSTD = 50000
11858            WEBP = 50001
11859            JPEGXL = 50002  # JXL
11860            PIXTIFF = 50013
11861            # EER_V0 = 65000
11862            # EER_V1 = 65001
11863            # KODAK_DCR = 65000
11864            # PENTAX_PEF = 65535
11865
11866            def __bool__(self):
11867                return self != 1
11868
11869        return COMPRESSION
11870
11871    def PHOTOMETRIC():
11872        class PHOTOMETRIC(enum.IntEnum):
11873            MINISWHITE = 0
11874            MINISBLACK = 1
11875            RGB = 2
11876            PALETTE = 3
11877            MASK = 4
11878            SEPARATED = 5  # CMYK
11879            YCBCR = 6
11880            CIELAB = 8
11881            ICCLAB = 9
11882            ITULAB = 10
11883            CFA = 32803  # Color Filter Array
11884            LOGL = 32844
11885            LOGLUV = 32845
11886            LINEAR_RAW = 34892
11887            DEPTH_MAP = 51177  # DNG 1.5
11888            SEMANTIC_MASK = 52527  # DNG 1.6
11889
11890        return PHOTOMETRIC
11891
11892    def PHOTOMETRIC_SAMPLES():
11893        return {
11894            0: 1,  # MINISWHITE
11895            1: 1,  # MINISBLACK
11896            2: 3,  # RGB
11897            3: 1,  # PALETTE
11898            4: 1,  # MASK
11899            5: 4,  # SEPARATED
11900            6: 3,  # YCBCR
11901            8: 3,  # CIELAB
11902            9: 3,  # ICCLAB
11903            10: 3,  # ITULAB
11904            32803: 1,  # CFA
11905            32844: 1,  # LOGL ?
11906            32845: 3,  # LOGLUV
11907            34892: 3,  # LINEAR_RAW ?
11908            51177: 1,  # DEPTH_MAP ?
11909            52527: 1,  # SEMANTIC_MASK ?
11910        }
11911
11912    def THRESHHOLD():
11913        class THRESHHOLD(enum.IntEnum):
11914            BILEVEL = 1
11915            HALFTONE = 2
11916            ERRORDIFFUSE = 3
11917
11918        return THRESHHOLD
11919
11920    def FILLORDER():
11921        class FILLORDER(enum.IntEnum):
11922            MSB2LSB = 1
11923            LSB2MSB = 2
11924
11925        return FILLORDER
11926
11927    def ORIENTATION():
11928        class ORIENTATION(enum.IntEnum):
11929            TOPLEFT = 1
11930            TOPRIGHT = 2
11931            BOTRIGHT = 3
11932            BOTLEFT = 4
11933            LEFTTOP = 5
11934            RIGHTTOP = 6
11935            RIGHTBOT = 7
11936            LEFTBOT = 8
11937
11938        return ORIENTATION
11939
11940    def PLANARCONFIG():
11941        class PLANARCONFIG(enum.IntEnum):
11942            CONTIG = 1  # CHUNKY
11943            SEPARATE = 2
11944
11945        return PLANARCONFIG
11946
11947    def GRAYRESPONSEUNIT():
11948        class GRAYRESPONSEUNIT(enum.IntEnum):
11949            _10S = 1
11950            _100S = 2
11951            _1000S = 3
11952            _10000S = 4
11953            _100000S = 5
11954
11955        return GRAYRESPONSEUNIT
11956
11957    def GROUP4OPT():
11958        class GROUP4OPT(enum.IntEnum):
11959            UNCOMPRESSED = 2
11960
11961        return GROUP4OPT
11962
11963    def RESUNIT():
11964        class RESUNIT(enum.IntEnum):
11965            NONE = 1
11966            INCH = 2
11967            CENTIMETER = 3
11968            MILLIMETER = 4  # DNG
11969            MICROMETER = 5  # DNG
11970
11971            def __bool__(self):
11972                return self != 1
11973
11974        return RESUNIT
11975
11976    def COLORRESPONSEUNIT():
11977        class COLORRESPONSEUNIT(enum.IntEnum):
11978            _10S = 1
11979            _100S = 2
11980            _1000S = 3
11981            _10000S = 4
11982            _100000S = 5
11983
11984        return COLORRESPONSEUNIT
11985
11986    def PREDICTOR():
11987        class PREDICTOR(enum.IntEnum):
11988            NONE = 1
11989            HORIZONTAL = 2
11990            FLOATINGPOINT = 3
11991            HORIZONTALX2 = 34892  # DNG
11992            HORIZONTALX4 = 34893
11993            FLOATINGPOINTX2 = 34894
11994            FLOATINGPOINTX4 = 34895
11995
11996            def __bool__(self):
11997                return self != 1
11998
11999        return PREDICTOR
12000
12001    def EXTRASAMPLE():
12002        class EXTRASAMPLE(enum.IntEnum):
12003            UNSPECIFIED = 0
12004            ASSOCALPHA = 1
12005            UNASSALPHA = 2
12006
12007        return EXTRASAMPLE
12008
12009    def SAMPLEFORMAT():
12010        class SAMPLEFORMAT(enum.IntEnum):
12011            UINT = 1
12012            INT = 2
12013            IEEEFP = 3
12014            VOID = 4
12015            COMPLEXINT = 5
12016            COMPLEXIEEEFP = 6
12017
12018        return SAMPLEFORMAT
12019
12020    def DATATYPES():
12021        class DATATYPES(enum.IntEnum):
12022            BYTE = 1  # 8-bit unsigned integer
12023            ASCII = 2  # 8-bit byte that contains a 7-bit ASCII code;
12024            #            the last byte must be NULL (binary zero)
12025            SHORT = 3  # 16-bit (2-byte) unsigned integer
12026            LONG = 4  # 32-bit (4-byte) unsigned integer
12027            RATIONAL = 5  # two LONGs: the first represents the numerator
12028            #               of a fraction; the second, the denominator
12029            SBYTE = 6  # an 8-bit signed (twos-complement) integer
12030            UNDEFINED = 7  # an 8-bit byte that may contain anything,
12031            #                depending on the definition of the field
12032            SSHORT = 8  # A 16-bit (2-byte) signed (twos-complement) integer
12033            SLONG = 9  # a 32-bit (4-byte) signed (twos-complement) integer
12034            SRATIONAL = 10  # two SLONGs: the first represents the numerator
12035            #                 of a fraction, the second the denominator
12036            FLOAT = 11  # single precision (4-byte) IEEE format
12037            DOUBLE = 12  # double precision (8-byte) IEEE format
12038            IFD = 13  # unsigned 4 byte IFD offset
12039            UNICODE = 14
12040            COMPLEX = 15
12041            LONG8 = 16  # unsigned 8 byte integer (BigTiff)
12042            SLONG8 = 17  # signed 8 byte integer (BigTiff)
12043            IFD8 = 18  # unsigned 8 byte IFD offset (BigTiff)
12044
12045        return DATATYPES
12046
12047    def DATA_FORMATS():
12048        # map TIFF DATATYPES to Python struct formats
12049        return {
12050            1: '1B',
12051            2: '1s',
12052            3: '1H',
12053            4: '1I',
12054            5: '2I',
12055            6: '1b',
12056            7: '1B',
12057            8: '1h',
12058            9: '1i',
12059            10: '2i',
12060            11: '1f',
12061            12: '1d',
12062            13: '1I',
12063            # 14: '',
12064            # 15: '',
12065            16: '1Q',
12066            17: '1q',
12067            18: '1Q',
12068        }
12069
12070    def DATA_DTYPES():
12071        # map numpy dtypes to TIFF DATATYPES
12072        return {
12073            'B': 1,
12074            's': 2,
12075            'H': 3,
12076            'I': 4,
12077            '2I': 5,
12078            'b': 6,
12079            'h': 8,
12080            'i': 9,
12081            '2i': 10,
12082            'f': 11,
12083            'd': 12,
12084            'Q': 16,
12085            'q': 17,
12086        }
12087
12088    def SAMPLE_DTYPES():
12089        # map SampleFormat and BitsPerSample to numpy dtype
12090        return {
12091            # UINT
12092            (1, 1): '?',  # bitmap
12093            (1, 2): 'B',
12094            (1, 3): 'B',
12095            (1, 4): 'B',
12096            (1, 5): 'B',
12097            (1, 6): 'B',
12098            (1, 7): 'B',
12099            (1, 8): 'B',
12100            (1, 9): 'H',
12101            (1, 10): 'H',
12102            (1, 11): 'H',
12103            (1, 12): 'H',
12104            (1, 13): 'H',
12105            (1, 14): 'H',
12106            (1, 15): 'H',
12107            (1, 16): 'H',
12108            (1, 17): 'I',
12109            (1, 18): 'I',
12110            (1, 19): 'I',
12111            (1, 20): 'I',
12112            (1, 21): 'I',
12113            (1, 22): 'I',
12114            (1, 23): 'I',
12115            (1, 24): 'I',
12116            (1, 25): 'I',
12117            (1, 26): 'I',
12118            (1, 27): 'I',
12119            (1, 28): 'I',
12120            (1, 29): 'I',
12121            (1, 30): 'I',
12122            (1, 31): 'I',
12123            (1, 32): 'I',
12124            (1, 64): 'Q',
12125            # VOID : treat as UINT
12126            (4, 1): '?',  # bitmap
12127            (4, 2): 'B',
12128            (4, 3): 'B',
12129            (4, 4): 'B',
12130            (4, 5): 'B',
12131            (4, 6): 'B',
12132            (4, 7): 'B',
12133            (4, 8): 'B',
12134            (4, 9): 'H',
12135            (4, 10): 'H',
12136            (4, 11): 'H',
12137            (4, 12): 'H',
12138            (4, 13): 'H',
12139            (4, 14): 'H',
12140            (4, 15): 'H',
12141            (4, 16): 'H',
12142            (4, 17): 'I',
12143            (4, 18): 'I',
12144            (4, 19): 'I',
12145            (4, 20): 'I',
12146            (4, 21): 'I',
12147            (4, 22): 'I',
12148            (4, 23): 'I',
12149            (4, 24): 'I',
12150            (4, 25): 'I',
12151            (4, 26): 'I',
12152            (4, 27): 'I',
12153            (4, 28): 'I',
12154            (4, 29): 'I',
12155            (4, 30): 'I',
12156            (4, 31): 'I',
12157            (4, 32): 'I',
12158            (4, 64): 'Q',
12159            # INT
12160            (2, 8): 'b',
12161            (2, 16): 'h',
12162            (2, 32): 'i',
12163            (2, 64): 'q',
12164            # IEEEFP
12165            (3, 16): 'e',
12166            (3, 24): 'f',  # float24 bit not supported by numpy
12167            (3, 32): 'f',
12168            (3, 64): 'd',
12169            # COMPLEXIEEEFP
12170            (6, 64): 'F',
12171            (6, 128): 'D',
12172            # RGB565
12173            (1, (5, 6, 5)): 'B',
12174            # COMPLEXINT : not supported by numpy
12175            (5, 16): 'E',
12176            (5, 32): 'F',
12177            (5, 64): 'D',
12178        }
12179
12180    def PREDICTORS():
12181        # map PREDICTOR to predictor encode functions
12182
12183        class PREDICTORS:
12184            def __init__(self):
12185                self._codecs = {None: identityfunc, 1: identityfunc}
12186                if imagecodecs is None:
12187                    self._codecs[2] = delta_encode
12188
12189            def __getitem__(self, key):
12190                if key in self._codecs:
12191                    return self._codecs[key]
12192                try:
12193                    if key == 2:
12194                        codec = imagecodecs.delta_encode
12195                    elif key == 3:
12196                        codec = imagecodecs.floatpred_encode
12197                    elif key == 34892:
12198
12199                        def codec(data, axis=-1, out=None):
12200                            return imagecodecs.delta_encode(
12201                                data, axis=axis, out=out, dist=2
12202                            )
12203
12204                    elif key == 34893:
12205
12206                        def codec(data, axis=-1, out=None):
12207                            return imagecodecs.delta_encode(
12208                                data, axis=axis, out=out, dist=4
12209                            )
12210
12211                    elif key == 34894:
12212
12213                        def codec(data, axis=-1, out=None):
12214                            return imagecodecs.floatpred_encode(
12215                                data, axis=axis, out=out, dist=2
12216                            )
12217
12218                    elif key == 34895:
12219
12220                        def codec(data, axis=-1, out=None):
12221                            return imagecodecs.floatpred_encode(
12222                                data, axis=axis, out=out, dist=4
12223                            )
12224
12225                    else:
12226                        raise KeyError(f'{key} is not a known PREDICTOR')
12227                except AttributeError:
12228                    raise KeyError(
12229                        f'{TIFF.PREDICTOR(key)!r}'
12230                        " requires the 'imagecodecs' package"
12231                    )
12232                self._codecs[key] = codec
12233                return codec
12234
12235        return PREDICTORS()
12236
12237    def UNPREDICTORS():
12238        # map PREDICTOR to predictor decode functions
12239
12240        class UNPREDICTORS:
12241            def __init__(self):
12242                self._codecs = {None: identityfunc, 1: identityfunc}
12243                if imagecodecs is None:
12244                    self._codecs[2] = delta_decode
12245
12246            def __getitem__(self, key):
12247                if key in self._codecs:
12248                    return self._codecs[key]
12249                try:
12250                    if key == 2:
12251                        codec = imagecodecs.delta_decode
12252                    elif key == 3:
12253                        codec = imagecodecs.floatpred_decode
12254                    elif key == 34892:
12255
12256                        def codec(data, axis=-1, out=None):
12257                            return imagecodecs.delta_decode(
12258                                data, axis=axis, out=out, dist=2
12259                            )
12260
12261                    elif key == 34893:
12262
12263                        def codec(data, axis=-1, out=None):
12264                            return imagecodecs.delta_decode(
12265                                data, axis=axis, out=out, dist=4
12266                            )
12267
12268                    elif key == 34894:
12269
12270                        def codec(data, axis=-1, out=None):
12271                            return imagecodecs.floatpred_decode(
12272                                data, axis=axis, out=out, dist=2
12273                            )
12274
12275                    elif key == 34895:
12276
12277                        def codec(data, axis=-1, out=None):
12278                            return imagecodecs.floatpred_decode(
12279                                data, axis=axis, out=out, dist=4
12280                            )
12281
12282                    else:
12283                        raise KeyError(f'{key} is not a known PREDICTOR')
12284                except AttributeError:
12285                    raise KeyError(
12286                        f'{TIFF.PREDICTOR(key)!r}'
12287                        " requires the 'imagecodecs' package"
12288                    )
12289                self._codecs[key] = codec
12290                return codec
12291
12292        return UNPREDICTORS()
12293
12294    def COMPRESSORS():
12295        # map COMPRESSION to compress functions
12296
12297        class COMPRESSORS:
12298            def __init__(self):
12299                self._codecs = {None: identityfunc, 1: identityfunc}
12300                if imagecodecs is None:
12301                    self._codecs[8] = zlib_encode
12302                    self._codecs[32946] = zlib_encode
12303                    self._codecs[34925] = lzma_encode
12304
12305            def __getitem__(self, key):
12306                if key in self._codecs:
12307                    return self._codecs[key]
12308                try:
12309                    if key == 5:
12310                        codec = imagecodecs.lzw_encode
12311                    elif key == 7:
12312                        codec = imagecodecs.jpeg_encode
12313                    elif key == 8 or key == 32946:
12314                        if (
12315                            hasattr(imagecodecs, 'DEFLATE')
12316                            and imagecodecs.DEFLATE
12317                        ):
12318                            codec = imagecodecs.deflate_encode
12319                        elif imagecodecs.ZLIB:
12320                            codec = imagecodecs.zlib_encode
12321                        else:
12322                            codec = zlib_encode
12323                    elif key == 32773:
12324                        codec = imagecodecs.packbits_encode
12325                    elif (
12326                        key == 33003
12327                        or key == 33004
12328                        or key == 33005
12329                        or key == 34712
12330                    ):
12331                        codec = imagecodecs.jpeg2k_encode
12332                    elif key == 34887:
12333                        codec = imagecodecs.lerc_encode
12334                    elif key == 34892:
12335                        codec = imagecodecs.jpeg8_encode  # DNG lossy
12336                    elif key == 34925:
12337                        if imagecodecs.LZMA:
12338                            codec = imagecodecs.lzma_encode
12339                        else:
12340                            codec = lzma_encode
12341                    elif key == 34933:
12342                        codec = imagecodecs.png_encode
12343                    elif key == 34934 or key == 22610:
12344                        codec = imagecodecs.jpegxr_encode
12345                    elif key == 50000:
12346                        codec = imagecodecs.zstd_encode
12347                    elif key == 50001:
12348                        codec = imagecodecs.webp_encode
12349                    elif key == 50002:
12350                        codec = imagecodecs.jpegxl_encode
12351                    else:
12352                        try:
12353                            msg = f'{TIFF.COMPRESSION(key)!r} not supported'
12354                        except ValueError:
12355                            msg = f'{key} is not a known COMPRESSION'
12356                        raise KeyError(msg)
12357                except AttributeError:
12358                    raise KeyError(
12359                        f'{TIFF.COMPRESSION(key)!r} '
12360                        "requires the 'imagecodecs' package"
12361                    )
12362                self._codecs[key] = codec
12363                return codec
12364
12365        return COMPRESSORS()
12366
12367    def DECOMPRESSORS():
12368        # map COMPRESSION to decompress functions
12369
12370        class DECOMPRESSORS:
12371            def __init__(self):
12372                self._codecs = {None: identityfunc, 1: identityfunc}
12373                if imagecodecs is None:
12374                    self._codecs[8] = zlib_decode
12375                    self._codecs[32773] = packbits_decode
12376                    self._codecs[32946] = zlib_decode
12377                    self._codecs[34925] = lzma_decode
12378
12379            def __getitem__(self, key):
12380                if key in self._codecs:
12381                    return self._codecs[key]
12382                try:
12383                    if key == 5:
12384                        codec = imagecodecs.lzw_decode
12385                    elif key == 6 or key == 7 or key == 33007:
12386                        codec = imagecodecs.jpeg_decode
12387                    elif key == 8 or key == 32946:
12388                        if (
12389                            hasattr(imagecodecs, 'DEFLATE')
12390                            and imagecodecs.DEFLATE
12391                        ):
12392                            codec = imagecodecs.deflate_decode
12393                        elif imagecodecs.ZLIB:
12394                            codec = imagecodecs.zlib_decode
12395                        else:
12396                            codec = zlib_decode
12397                    elif key == 32773:
12398                        codec = imagecodecs.packbits_decode
12399                    elif (
12400                        key == 33003
12401                        or key == 33004
12402                        or key == 33005
12403                        or key == 34712
12404                    ):
12405                        codec = imagecodecs.jpeg2k_decode
12406                    elif key == 34887:
12407                        codec = imagecodecs.lerc_decode
12408                    elif key == 34892:
12409                        codec = imagecodecs.jpeg8_decode  # DNG lossy
12410                    elif key == 34925:
12411                        if imagecodecs.LZMA:
12412                            codec = imagecodecs.lzma_decode
12413                        else:
12414                            codec = lzma_decode
12415                    elif key == 34933:
12416                        codec = imagecodecs.png_decode
12417                    elif key == 34934 or key == 22610:
12418                        codec = imagecodecs.jpegxr_decode
12419                    elif key == 50000 or key == 34926:  # 34926 deprecated
12420                        codec = imagecodecs.zstd_decode
12421                    elif key == 50001 or key == 34927:  # 34927 deprecated
12422                        codec = imagecodecs.webp_decode
12423                    elif key == 50002:
12424                        codec = imagecodecs.jpegxl_decode
12425                    else:
12426                        try:
12427                            msg = f'{TIFF.COMPRESSION(key)!r} not supported'
12428                        except ValueError:
12429                            msg = f'{key} is not a known COMPRESSION'
12430                        raise KeyError(msg)
12431                except AttributeError:
12432                    raise KeyError(
12433                        f'{TIFF.COMPRESSION(key)!r} '
12434                        "requires the 'imagecodecs' package"
12435                    )
12436                self._codecs[key] = codec
12437                return codec
12438
12439            def __contains__(self, key):
12440                try:
12441                    self[key]
12442                except KeyError:
12443                    return False
12444                return True
12445
12446        return DECOMPRESSORS()
12447
12448    def FRAME_ATTRS():
12449        # attributes that a TiffFrame shares with its keyframe
12450        return {'shape', 'ndim', 'size', 'dtype', 'axes', 'is_final', 'decode'}
12451
12452    def FILE_FLAGS():
12453        # TiffFile and TiffPage 'is_\*' attributes
12454        exclude = {
12455            'reduced',
12456            'mask',
12457            'final',
12458            'memmappable',
12459            'contiguous',
12460            'tiled',
12461            'subsampled',
12462        }
12463        return {
12464            a[3:]
12465            for a in dir(TiffPage)
12466            if a[:3] == 'is_' and a[3:] not in exclude
12467        }
12468
12469    def FILE_PATTERNS():
12470        # predefined FileSequence patterns
12471        return {
12472            'axes': r"""(?ix)
12473                # matches Olympus OIF and Leica TIFF series
12474                _?(?:(q|l|p|a|c|t|x|y|z|ch|tp)(\d{1,4}))
12475                _?(?:(q|l|p|a|c|t|x|y|z|ch|tp)(\d{1,4}))?
12476                _?(?:(q|l|p|a|c|t|x|y|z|ch|tp)(\d{1,4}))?
12477                _?(?:(q|l|p|a|c|t|x|y|z|ch|tp)(\d{1,4}))?
12478                _?(?:(q|l|p|a|c|t|x|y|z|ch|tp)(\d{1,4}))?
12479                _?(?:(q|l|p|a|c|t|x|y|z|ch|tp)(\d{1,4}))?
12480                _?(?:(q|l|p|a|c|t|x|y|z|ch|tp)(\d{1,4}))?
12481                """
12482        }
12483
12484    def FILE_EXTENSIONS():
12485        # TIFF file extensions
12486        return (
12487            'tif',
12488            'tiff',
12489            'ome.tif',
12490            'lsm',
12491            'stk',
12492            'qpi',
12493            'pcoraw',
12494            'qptiff',
12495            'gel',
12496            'seq',
12497            'svs',
12498            'scn',
12499            'zif',
12500            'ndpi',
12501            'bif',
12502            'tf8',
12503            'tf2',
12504            'btf',
12505            'eer',
12506        )
12507
12508    def FILEOPEN_FILTER():
12509        # string for use in Windows File Open box
12510        return [
12511            (f'{ext.upper()} files', f'*.{ext}')
12512            for ext in TIFF.FILE_EXTENSIONS
12513        ] + [('allfiles', '*')]
12514
12515    def AXES_LABELS():
12516
12517        # TODO: is there a standard for character axes labels?
12518        axes = {
12519            'X': 'width',
12520            'Y': 'length',  # height
12521            'Z': 'depth',
12522            'S': 'sample',  # rgb(a), cmyk
12523            'I': 'series',  # general sequence of frames/planes/pages/IFDs
12524            'T': 'time',
12525            'C': 'channel',  # color, emission wavelength
12526            'A': 'angle',
12527            'P': 'phase',  # formerly F    # P is Position in LSM!
12528            'R': 'tile',  # region, point, mosaic
12529            'H': 'lifetime',  # histogram
12530            'E': 'lambda',  # excitation wavelength
12531            'L': 'exposure',  # lux
12532            'V': 'event',
12533            'Q': 'other',
12534            'M': 'mosaic',  # LSM 6
12535        }
12536        axes.update({v: k for k, v in axes.items()})
12537        return axes
12538
12539    def NDPI_TAGS():
12540        # 65420 - 65458  Private Hamamatsu NDPI tags
12541        # TODO: obtain specification
12542        return TiffTagRegistry(
12543            (
12544                (65324, 'OffsetHighBytes'),
12545                (65325, 'ByteCountHighBytes'),
12546                (65420, 'FileFormat'),
12547                (65421, 'Magnification'),  # SourceLens
12548                (65422, 'XOffsetFromSlideCenter'),
12549                (65423, 'YOffsetFromSlideCenter'),
12550                (65424, 'ZOffsetFromSlideCenter'),  # FocalPlane
12551                (65425, 'TissueIndex'),
12552                (65426, 'McuStarts'),
12553                (65427, 'SlideLabel'),
12554                (65428, 'AuthCode'),  # ?
12555                (65429, '65429'),
12556                (65430, '65430'),
12557                (65431, '65431'),
12558                (65432, 'McuStartsHighBytes'),
12559                (65433, '65433'),
12560                (65434, 'Fluorescence'),  # FilterSetName
12561                (65435, 'ExposureRatio'),
12562                (65436, 'RedMultiplier'),
12563                (65437, 'GreenMultiplier'),
12564                (65438, 'BlueMultiplier'),
12565                (65439, 'FocusPoints'),
12566                (65440, 'FocusPointRegions'),
12567                (65441, 'CaptureMode'),
12568                (65442, 'ScannerSerialNumber'),
12569                (65443, '65443'),
12570                (65444, 'JpegQuality'),
12571                (65445, 'RefocusInterval'),
12572                (65446, 'FocusOffset'),
12573                (65447, 'BlankLines'),
12574                (65448, 'FirmwareVersion'),
12575                (65449, 'Comments'),  # PropertyMap, CalibrationInfo
12576                (65450, 'LabelObscured'),
12577                (65451, 'Wavelength'),
12578                (65452, '65452'),
12579                (65453, 'LampAge'),
12580                (65454, 'ExposureTime'),
12581                (65455, 'FocusTime'),
12582                (65456, 'ScanTime'),
12583                (65457, 'WriteTime'),
12584                (65458, 'FullyAutoFocus'),
12585                (65500, 'DefaultGamma'),
12586            )
12587        )
12588
12589    def EXIF_TAGS():
12590        # 65000 - 65112  Photoshop Camera RAW EXIF tags
12591        tags = TiffTagRegistry(
12592            (
12593                (65000, 'OwnerName'),
12594                (65001, 'SerialNumber'),
12595                (65002, 'Lens'),
12596                (65100, 'RawFile'),
12597                (65101, 'Converter'),
12598                (65102, 'WhiteBalance'),
12599                (65105, 'Exposure'),
12600                (65106, 'Shadows'),
12601                (65107, 'Brightness'),
12602                (65108, 'Contrast'),
12603                (65109, 'Saturation'),
12604                (65110, 'Sharpness'),
12605                (65111, 'Smoothness'),
12606                (65112, 'MoireFilter'),
12607            )
12608        )
12609        tags.update(TIFF.TAGS)
12610        return tags
12611
12612    def GPS_TAGS():
12613        return TiffTagRegistry(
12614            (
12615                (0, 'GPSVersionID'),
12616                (1, 'GPSLatitudeRef'),
12617                (2, 'GPSLatitude'),
12618                (3, 'GPSLongitudeRef'),
12619                (4, 'GPSLongitude'),
12620                (5, 'GPSAltitudeRef'),
12621                (6, 'GPSAltitude'),
12622                (7, 'GPSTimeStamp'),
12623                (8, 'GPSSatellites'),
12624                (9, 'GPSStatus'),
12625                (10, 'GPSMeasureMode'),
12626                (11, 'GPSDOP'),
12627                (12, 'GPSSpeedRef'),
12628                (13, 'GPSSpeed'),
12629                (14, 'GPSTrackRef'),
12630                (15, 'GPSTrack'),
12631                (16, 'GPSImgDirectionRef'),
12632                (17, 'GPSImgDirection'),
12633                (18, 'GPSMapDatum'),
12634                (19, 'GPSDestLatitudeRef'),
12635                (20, 'GPSDestLatitude'),
12636                (21, 'GPSDestLongitudeRef'),
12637                (22, 'GPSDestLongitude'),
12638                (23, 'GPSDestBearingRef'),
12639                (24, 'GPSDestBearing'),
12640                (25, 'GPSDestDistanceRef'),
12641                (26, 'GPSDestDistance'),
12642                (27, 'GPSProcessingMethod'),
12643                (28, 'GPSAreaInformation'),
12644                (29, 'GPSDateStamp'),
12645                (30, 'GPSDifferential'),
12646                (31, 'GPSHPositioningError'),
12647            )
12648        )
12649
12650    def IOP_TAGS():
12651        return TiffTagRegistry(
12652            (
12653                (1, 'InteroperabilityIndex'),
12654                (2, 'InteroperabilityVersion'),
12655                (4096, 'RelatedImageFileFormat'),
12656                (4097, 'RelatedImageWidth'),
12657                (4098, 'RelatedImageLength'),
12658            )
12659        )
12660
12661    def GEO_KEYS():
12662        try:
12663            from .tifffile_geodb import GeoKeys  # delayed import
12664        except ImportError:
12665            try:
12666                from tifffile_geodb import GeoKeys  # delayed import
12667            except ImportError:
12668
12669                class GeoKeys(enum.IntEnum):
12670                    pass
12671
12672        return GeoKeys
12673
12674    def GEO_CODES():
12675        try:
12676            from .tifffile_geodb import GEO_CODES  # delayed import
12677        except ImportError:
12678            try:
12679                from tifffile_geodb import GEO_CODES  # delayed import
12680            except ImportError:
12681                GEO_CODES = {}
12682        return GEO_CODES
12683
12684    def CZ_LSMINFO():
12685        return [
12686            ('MagicNumber', 'u4'),
12687            ('StructureSize', 'i4'),
12688            ('DimensionX', 'i4'),
12689            ('DimensionY', 'i4'),
12690            ('DimensionZ', 'i4'),
12691            ('DimensionChannels', 'i4'),
12692            ('DimensionTime', 'i4'),
12693            ('DataType', 'i4'),  # DATATYPES
12694            ('ThumbnailX', 'i4'),
12695            ('ThumbnailY', 'i4'),
12696            ('VoxelSizeX', 'f8'),
12697            ('VoxelSizeY', 'f8'),
12698            ('VoxelSizeZ', 'f8'),
12699            ('OriginX', 'f8'),
12700            ('OriginY', 'f8'),
12701            ('OriginZ', 'f8'),
12702            ('ScanType', 'u2'),
12703            ('SpectralScan', 'u2'),
12704            ('TypeOfData', 'u4'),  # TYPEOFDATA
12705            ('OffsetVectorOverlay', 'u4'),
12706            ('OffsetInputLut', 'u4'),
12707            ('OffsetOutputLut', 'u4'),
12708            ('OffsetChannelColors', 'u4'),
12709            ('TimeIntervall', 'f8'),
12710            ('OffsetChannelDataTypes', 'u4'),
12711            ('OffsetScanInformation', 'u4'),  # SCANINFO
12712            ('OffsetKsData', 'u4'),
12713            ('OffsetTimeStamps', 'u4'),
12714            ('OffsetEventList', 'u4'),
12715            ('OffsetRoi', 'u4'),
12716            ('OffsetBleachRoi', 'u4'),
12717            ('OffsetNextRecording', 'u4'),
12718            # LSM 2.0 ends here
12719            ('DisplayAspectX', 'f8'),
12720            ('DisplayAspectY', 'f8'),
12721            ('DisplayAspectZ', 'f8'),
12722            ('DisplayAspectTime', 'f8'),
12723            ('OffsetMeanOfRoisOverlay', 'u4'),
12724            ('OffsetTopoIsolineOverlay', 'u4'),
12725            ('OffsetTopoProfileOverlay', 'u4'),
12726            ('OffsetLinescanOverlay', 'u4'),
12727            ('ToolbarFlags', 'u4'),
12728            ('OffsetChannelWavelength', 'u4'),
12729            ('OffsetChannelFactors', 'u4'),
12730            ('ObjectiveSphereCorrection', 'f8'),
12731            ('OffsetUnmixParameters', 'u4'),
12732            # LSM 3.2, 4.0 end here
12733            ('OffsetAcquisitionParameters', 'u4'),
12734            ('OffsetCharacteristics', 'u4'),
12735            ('OffsetPalette', 'u4'),
12736            ('TimeDifferenceX', 'f8'),
12737            ('TimeDifferenceY', 'f8'),
12738            ('TimeDifferenceZ', 'f8'),
12739            ('InternalUse1', 'u4'),
12740            ('DimensionP', 'i4'),
12741            ('DimensionM', 'i4'),
12742            ('DimensionsReserved', '16i4'),
12743            ('OffsetTilePositions', 'u4'),
12744            ('', '9u4'),  # Reserved
12745            ('OffsetPositions', 'u4'),
12746            # ('', '21u4'),  # must be 0
12747        ]
12748
12749    def CZ_LSMINFO_READERS():
12750        # import functions for CZ_LSMINFO sub-records
12751        # TODO: read more CZ_LSMINFO sub-records
12752        return {
12753            'ScanInformation': read_lsm_scaninfo,
12754            'TimeStamps': read_lsm_timestamps,
12755            'EventList': read_lsm_eventlist,
12756            'ChannelColors': read_lsm_channelcolors,
12757            'Positions': read_lsm_positions,
12758            'TilePositions': read_lsm_positions,
12759            'VectorOverlay': None,
12760            'InputLut': read_lsm_lookuptable,
12761            'OutputLut': read_lsm_lookuptable,
12762            'TimeIntervall': None,
12763            'ChannelDataTypes': read_lsm_channeldatatypes,
12764            'KsData': None,
12765            'Roi': None,
12766            'BleachRoi': None,
12767            'NextRecording': None,  # read with TiffFile(fh, offset=)
12768            'MeanOfRoisOverlay': None,
12769            'TopoIsolineOverlay': None,
12770            'TopoProfileOverlay': None,
12771            'ChannelWavelength': read_lsm_channelwavelength,
12772            'SphereCorrection': None,
12773            'ChannelFactors': None,
12774            'UnmixParameters': None,
12775            'AcquisitionParameters': None,
12776            'Characteristics': None,
12777        }
12778
12779    def CZ_LSMINFO_SCANTYPE():
12780        # map CZ_LSMINFO.ScanType to dimension order
12781        return {
12782            0: 'XYZCT',  # 'Stack' normal x-y-z-scan
12783            1: 'XYZCT',  # 'Z-Scan' x-z-plane Y=1
12784            2: 'XYZCT',  # 'Line'
12785            3: 'XYTCZ',  # 'Time Series Plane' time series x-y  XYCTZ ? Z=1
12786            4: 'XYZTC',  # 'Time Series z-Scan' time series x-z
12787            5: 'XYTCZ',  # 'Time Series Mean-of-ROIs'
12788            6: 'XYZTC',  # 'Time Series Stack' time series x-y-z
12789            7: 'XYCTZ',  # Spline Scan
12790            8: 'XYCZT',  # Spline Plane x-z
12791            9: 'XYTCZ',  # Time Series Spline Plane x-z
12792            10: 'XYZCT',  # 'Time Series Point' point mode
12793        }
12794
12795    def CZ_LSMINFO_DIMENSIONS():
12796        # map dimension codes to CZ_LSMINFO attribute
12797        return {
12798            'X': 'DimensionX',
12799            'Y': 'DimensionY',
12800            'Z': 'DimensionZ',
12801            'C': 'DimensionChannels',
12802            'T': 'DimensionTime',
12803            'P': 'DimensionP',
12804            'M': 'DimensionM',
12805        }
12806
12807    def CZ_LSMINFO_DATATYPES():
12808        # description of CZ_LSMINFO.DataType
12809        return {
12810            0: 'varying data types',
12811            1: '8 bit unsigned integer',
12812            2: '12 bit unsigned integer',
12813            5: '32 bit float',
12814        }
12815
12816    def CZ_LSMINFO_TYPEOFDATA():
12817        # description of CZ_LSMINFO.TypeOfData
12818        return {
12819            0: 'Original scan data',
12820            1: 'Calculated data',
12821            2: '3D reconstruction',
12822            3: 'Topography height map',
12823        }
12824
12825    def CZ_LSMINFO_SCANINFO_ARRAYS():
12826        return {
12827            0x20000000: 'Tracks',
12828            0x30000000: 'Lasers',
12829            0x60000000: 'DetectionChannels',
12830            0x80000000: 'IlluminationChannels',
12831            0xA0000000: 'BeamSplitters',
12832            0xC0000000: 'DataChannels',
12833            0x11000000: 'Timers',
12834            0x13000000: 'Markers',
12835        }
12836
12837    def CZ_LSMINFO_SCANINFO_STRUCTS():
12838        return {
12839            # 0x10000000: 'Recording',
12840            0x40000000: 'Track',
12841            0x50000000: 'Laser',
12842            0x70000000: 'DetectionChannel',
12843            0x90000000: 'IlluminationChannel',
12844            0xB0000000: 'BeamSplitter',
12845            0xD0000000: 'DataChannel',
12846            0x12000000: 'Timer',
12847            0x14000000: 'Marker',
12848        }
12849
12850    def CZ_LSMINFO_SCANINFO_ATTRIBUTES():
12851        return {
12852            # Recording
12853            0x10000001: 'Name',
12854            0x10000002: 'Description',
12855            0x10000003: 'Notes',
12856            0x10000004: 'Objective',
12857            0x10000005: 'ProcessingSummary',
12858            0x10000006: 'SpecialScanMode',
12859            0x10000007: 'ScanType',
12860            0x10000008: 'ScanMode',
12861            0x10000009: 'NumberOfStacks',
12862            0x1000000A: 'LinesPerPlane',
12863            0x1000000B: 'SamplesPerLine',
12864            0x1000000C: 'PlanesPerVolume',
12865            0x1000000D: 'ImagesWidth',
12866            0x1000000E: 'ImagesHeight',
12867            0x1000000F: 'ImagesNumberPlanes',
12868            0x10000010: 'ImagesNumberStacks',
12869            0x10000011: 'ImagesNumberChannels',
12870            0x10000012: 'LinscanXySize',
12871            0x10000013: 'ScanDirection',
12872            0x10000014: 'TimeSeries',
12873            0x10000015: 'OriginalScanData',
12874            0x10000016: 'ZoomX',
12875            0x10000017: 'ZoomY',
12876            0x10000018: 'ZoomZ',
12877            0x10000019: 'Sample0X',
12878            0x1000001A: 'Sample0Y',
12879            0x1000001B: 'Sample0Z',
12880            0x1000001C: 'SampleSpacing',
12881            0x1000001D: 'LineSpacing',
12882            0x1000001E: 'PlaneSpacing',
12883            0x1000001F: 'PlaneWidth',
12884            0x10000020: 'PlaneHeight',
12885            0x10000021: 'VolumeDepth',
12886            0x10000023: 'Nutation',
12887            0x10000034: 'Rotation',
12888            0x10000035: 'Precession',
12889            0x10000036: 'Sample0time',
12890            0x10000037: 'StartScanTriggerIn',
12891            0x10000038: 'StartScanTriggerOut',
12892            0x10000039: 'StartScanEvent',
12893            0x10000040: 'StartScanTime',
12894            0x10000041: 'StopScanTriggerIn',
12895            0x10000042: 'StopScanTriggerOut',
12896            0x10000043: 'StopScanEvent',
12897            0x10000044: 'StopScanTime',
12898            0x10000045: 'UseRois',
12899            0x10000046: 'UseReducedMemoryRois',
12900            0x10000047: 'User',
12901            0x10000048: 'UseBcCorrection',
12902            0x10000049: 'PositionBcCorrection1',
12903            0x10000050: 'PositionBcCorrection2',
12904            0x10000051: 'InterpolationY',
12905            0x10000052: 'CameraBinning',
12906            0x10000053: 'CameraSupersampling',
12907            0x10000054: 'CameraFrameWidth',
12908            0x10000055: 'CameraFrameHeight',
12909            0x10000056: 'CameraOffsetX',
12910            0x10000057: 'CameraOffsetY',
12911            0x10000059: 'RtBinning',
12912            0x1000005A: 'RtFrameWidth',
12913            0x1000005B: 'RtFrameHeight',
12914            0x1000005C: 'RtRegionWidth',
12915            0x1000005D: 'RtRegionHeight',
12916            0x1000005E: 'RtOffsetX',
12917            0x1000005F: 'RtOffsetY',
12918            0x10000060: 'RtZoom',
12919            0x10000061: 'RtLinePeriod',
12920            0x10000062: 'Prescan',
12921            0x10000063: 'ScanDirectionZ',
12922            # Track
12923            0x40000001: 'MultiplexType',  # 0 After Line; 1 After Frame
12924            0x40000002: 'MultiplexOrder',
12925            0x40000003: 'SamplingMode',  # 0 Sample; 1 Line Avg; 2 Frame Avg
12926            0x40000004: 'SamplingMethod',  # 1 Mean; 2 Sum
12927            0x40000005: 'SamplingNumber',
12928            0x40000006: 'Acquire',
12929            0x40000007: 'SampleObservationTime',
12930            0x4000000B: 'TimeBetweenStacks',
12931            0x4000000C: 'Name',
12932            0x4000000D: 'Collimator1Name',
12933            0x4000000E: 'Collimator1Position',
12934            0x4000000F: 'Collimator2Name',
12935            0x40000010: 'Collimator2Position',
12936            0x40000011: 'IsBleachTrack',
12937            0x40000012: 'IsBleachAfterScanNumber',
12938            0x40000013: 'BleachScanNumber',
12939            0x40000014: 'TriggerIn',
12940            0x40000015: 'TriggerOut',
12941            0x40000016: 'IsRatioTrack',
12942            0x40000017: 'BleachCount',
12943            0x40000018: 'SpiCenterWavelength',
12944            0x40000019: 'PixelTime',
12945            0x40000021: 'CondensorFrontlens',
12946            0x40000023: 'FieldStopValue',
12947            0x40000024: 'IdCondensorAperture',
12948            0x40000025: 'CondensorAperture',
12949            0x40000026: 'IdCondensorRevolver',
12950            0x40000027: 'CondensorFilter',
12951            0x40000028: 'IdTransmissionFilter1',
12952            0x40000029: 'IdTransmission1',
12953            0x40000030: 'IdTransmissionFilter2',
12954            0x40000031: 'IdTransmission2',
12955            0x40000032: 'RepeatBleach',
12956            0x40000033: 'EnableSpotBleachPos',
12957            0x40000034: 'SpotBleachPosx',
12958            0x40000035: 'SpotBleachPosy',
12959            0x40000036: 'SpotBleachPosz',
12960            0x40000037: 'IdTubelens',
12961            0x40000038: 'IdTubelensPosition',
12962            0x40000039: 'TransmittedLight',
12963            0x4000003A: 'ReflectedLight',
12964            0x4000003B: 'SimultanGrabAndBleach',
12965            0x4000003C: 'BleachPixelTime',
12966            # Laser
12967            0x50000001: 'Name',
12968            0x50000002: 'Acquire',
12969            0x50000003: 'Power',
12970            # DetectionChannel
12971            0x70000001: 'IntegrationMode',
12972            0x70000002: 'SpecialMode',
12973            0x70000003: 'DetectorGainFirst',
12974            0x70000004: 'DetectorGainLast',
12975            0x70000005: 'AmplifierGainFirst',
12976            0x70000006: 'AmplifierGainLast',
12977            0x70000007: 'AmplifierOffsFirst',
12978            0x70000008: 'AmplifierOffsLast',
12979            0x70000009: 'PinholeDiameter',
12980            0x7000000A: 'CountingTrigger',
12981            0x7000000B: 'Acquire',
12982            0x7000000C: 'PointDetectorName',
12983            0x7000000D: 'AmplifierName',
12984            0x7000000E: 'PinholeName',
12985            0x7000000F: 'FilterSetName',
12986            0x70000010: 'FilterName',
12987            0x70000013: 'IntegratorName',
12988            0x70000014: 'ChannelName',
12989            0x70000015: 'DetectorGainBc1',
12990            0x70000016: 'DetectorGainBc2',
12991            0x70000017: 'AmplifierGainBc1',
12992            0x70000018: 'AmplifierGainBc2',
12993            0x70000019: 'AmplifierOffsetBc1',
12994            0x70000020: 'AmplifierOffsetBc2',
12995            0x70000021: 'SpectralScanChannels',
12996            0x70000022: 'SpiWavelengthStart',
12997            0x70000023: 'SpiWavelengthStop',
12998            0x70000026: 'DyeName',
12999            0x70000027: 'DyeFolder',
13000            # IlluminationChannel
13001            0x90000001: 'Name',
13002            0x90000002: 'Power',
13003            0x90000003: 'Wavelength',
13004            0x90000004: 'Aquire',
13005            0x90000005: 'DetchannelName',
13006            0x90000006: 'PowerBc1',
13007            0x90000007: 'PowerBc2',
13008            # BeamSplitter
13009            0xB0000001: 'FilterSet',
13010            0xB0000002: 'Filter',
13011            0xB0000003: 'Name',
13012            # DataChannel
13013            0xD0000001: 'Name',
13014            0xD0000003: 'Acquire',
13015            0xD0000004: 'Color',
13016            0xD0000005: 'SampleType',
13017            0xD0000006: 'BitsPerSample',
13018            0xD0000007: 'RatioType',
13019            0xD0000008: 'RatioTrack1',
13020            0xD0000009: 'RatioTrack2',
13021            0xD000000A: 'RatioChannel1',
13022            0xD000000B: 'RatioChannel2',
13023            0xD000000C: 'RatioConst1',
13024            0xD000000D: 'RatioConst2',
13025            0xD000000E: 'RatioConst3',
13026            0xD000000F: 'RatioConst4',
13027            0xD0000010: 'RatioConst5',
13028            0xD0000011: 'RatioConst6',
13029            0xD0000012: 'RatioFirstImages1',
13030            0xD0000013: 'RatioFirstImages2',
13031            0xD0000014: 'DyeName',
13032            0xD0000015: 'DyeFolder',
13033            0xD0000016: 'Spectrum',
13034            0xD0000017: 'Acquire',
13035            # Timer
13036            0x12000001: 'Name',
13037            0x12000002: 'Description',
13038            0x12000003: 'Interval',
13039            0x12000004: 'TriggerIn',
13040            0x12000005: 'TriggerOut',
13041            0x12000006: 'ActivationTime',
13042            0x12000007: 'ActivationNumber',
13043            # Marker
13044            0x14000001: 'Name',
13045            0x14000002: 'Description',
13046            0x14000003: 'TriggerIn',
13047            0x14000004: 'TriggerOut',
13048        }
13049
13050    def CZ_LSM_LUTTYPE():
13051        class CZ_LSM_LUTTYPE(enum.IntEnum):
13052            NORMAL = 0
13053            ORIGINAL = 1
13054            RAMP = 2
13055            POLYLINE = 3
13056            SPLINE = 4
13057            GAMMA = 5
13058
13059        return CZ_LSM_LUTTYPE
13060
13061    def CZ_LSM_SUBBLOCK_TYPE():
13062        class CZ_LSM_SUBBLOCK_TYPE(enum.IntEnum):
13063            END = 0
13064            GAMMA = 1
13065            BRIGHTNESS = 2
13066            CONTRAST = 3
13067            RAMP = 4
13068            KNOTS = 5
13069            PALETTE_12_TO_12 = 6
13070
13071        return CZ_LSM_SUBBLOCK_TYPE
13072
13073    def NIH_IMAGE_HEADER():
13074        return [
13075            ('FileID', 'a8'),
13076            ('nLines', 'i2'),
13077            ('PixelsPerLine', 'i2'),
13078            ('Version', 'i2'),
13079            ('OldLutMode', 'i2'),
13080            ('OldnColors', 'i2'),
13081            ('Colors', 'u1', (3, 32)),
13082            ('OldColorStart', 'i2'),
13083            ('ColorWidth', 'i2'),
13084            ('ExtraColors', 'u2', (6, 3)),
13085            ('nExtraColors', 'i2'),
13086            ('ForegroundIndex', 'i2'),
13087            ('BackgroundIndex', 'i2'),
13088            ('XScale', 'f8'),
13089            ('Unused2', 'i2'),
13090            ('Unused3', 'i2'),
13091            ('UnitsID', 'i2'),  # NIH_UNITS_TYPE
13092            ('p1', [('x', 'i2'), ('y', 'i2')]),
13093            ('p2', [('x', 'i2'), ('y', 'i2')]),
13094            ('CurveFitType', 'i2'),  # NIH_CURVEFIT_TYPE
13095            ('nCoefficients', 'i2'),
13096            ('Coeff', 'f8', 6),
13097            ('UMsize', 'u1'),
13098            ('UM', 'a15'),
13099            ('UnusedBoolean', 'u1'),
13100            ('BinaryPic', 'b1'),
13101            ('SliceStart', 'i2'),
13102            ('SliceEnd', 'i2'),
13103            ('ScaleMagnification', 'f4'),
13104            ('nSlices', 'i2'),
13105            ('SliceSpacing', 'f4'),
13106            ('CurrentSlice', 'i2'),
13107            ('FrameInterval', 'f4'),
13108            ('PixelAspectRatio', 'f4'),
13109            ('ColorStart', 'i2'),
13110            ('ColorEnd', 'i2'),
13111            ('nColors', 'i2'),
13112            ('Fill1', '3u2'),
13113            ('Fill2', '3u2'),
13114            ('Table', 'u1'),  # NIH_COLORTABLE_TYPE
13115            ('LutMode', 'u1'),  # NIH_LUTMODE_TYPE
13116            ('InvertedTable', 'b1'),
13117            ('ZeroClip', 'b1'),
13118            ('XUnitSize', 'u1'),
13119            ('XUnit', 'a11'),
13120            ('StackType', 'i2'),  # NIH_STACKTYPE_TYPE
13121            # ('UnusedBytes', 'u1', 200)
13122        ]
13123
13124    def NIH_COLORTABLE_TYPE():
13125        return (
13126            'CustomTable',
13127            'AppleDefault',
13128            'Pseudo20',
13129            'Pseudo32',
13130            'Rainbow',
13131            'Fire1',
13132            'Fire2',
13133            'Ice',
13134            'Grays',
13135            'Spectrum',
13136        )
13137
13138    def NIH_LUTMODE_TYPE():
13139        return (
13140            'PseudoColor',
13141            'OldAppleDefault',
13142            'OldSpectrum',
13143            'GrayScale',
13144            'ColorLut',
13145            'CustomGrayscale',
13146        )
13147
13148    def NIH_CURVEFIT_TYPE():
13149        return (
13150            'StraightLine',
13151            'Poly2',
13152            'Poly3',
13153            'Poly4',
13154            'Poly5',
13155            'ExpoFit',
13156            'PowerFit',
13157            'LogFit',
13158            'RodbardFit',
13159            'SpareFit1',
13160            'Uncalibrated',
13161            'UncalibratedOD',
13162        )
13163
13164    def NIH_UNITS_TYPE():
13165        return (
13166            'Nanometers',
13167            'Micrometers',
13168            'Millimeters',
13169            'Centimeters',
13170            'Meters',
13171            'Kilometers',
13172            'Inches',
13173            'Feet',
13174            'Miles',
13175            'Pixels',
13176            'OtherUnits',
13177        )
13178
13179    def TVIPS_HEADER_V1():
13180        # TVIPS TemData structure from EMMENU Help file
13181        return [
13182            ('Version', 'i4'),
13183            ('CommentV1', 'a80'),
13184            ('HighTension', 'i4'),
13185            ('SphericalAberration', 'i4'),
13186            ('IlluminationAperture', 'i4'),
13187            ('Magnification', 'i4'),
13188            ('PostMagnification', 'i4'),
13189            ('FocalLength', 'i4'),
13190            ('Defocus', 'i4'),
13191            ('Astigmatism', 'i4'),
13192            ('AstigmatismDirection', 'i4'),
13193            ('BiprismVoltage', 'i4'),
13194            ('SpecimenTiltAngle', 'i4'),
13195            ('SpecimenTiltDirection', 'i4'),
13196            ('IlluminationTiltDirection', 'i4'),
13197            ('IlluminationTiltAngle', 'i4'),
13198            ('ImageMode', 'i4'),
13199            ('EnergySpread', 'i4'),
13200            ('ChromaticAberration', 'i4'),
13201            ('ShutterType', 'i4'),
13202            ('DefocusSpread', 'i4'),
13203            ('CcdNumber', 'i4'),
13204            ('CcdSize', 'i4'),
13205            ('OffsetXV1', 'i4'),
13206            ('OffsetYV1', 'i4'),
13207            ('PhysicalPixelSize', 'i4'),
13208            ('Binning', 'i4'),
13209            ('ReadoutSpeed', 'i4'),
13210            ('GainV1', 'i4'),
13211            ('SensitivityV1', 'i4'),
13212            ('ExposureTimeV1', 'i4'),
13213            ('FlatCorrected', 'i4'),
13214            ('DeadPxCorrected', 'i4'),
13215            ('ImageMean', 'i4'),
13216            ('ImageStd', 'i4'),
13217            ('DisplacementX', 'i4'),
13218            ('DisplacementY', 'i4'),
13219            ('DateV1', 'i4'),
13220            ('TimeV1', 'i4'),
13221            ('ImageMin', 'i4'),
13222            ('ImageMax', 'i4'),
13223            ('ImageStatisticsQuality', 'i4'),
13224        ]
13225
13226    def TVIPS_HEADER_V2():
13227        return [
13228            ('ImageName', 'V160'),  # utf16
13229            ('ImageFolder', 'V160'),
13230            ('ImageSizeX', 'i4'),
13231            ('ImageSizeY', 'i4'),
13232            ('ImageSizeZ', 'i4'),
13233            ('ImageSizeE', 'i4'),
13234            ('ImageDataType', 'i4'),
13235            ('Date', 'i4'),
13236            ('Time', 'i4'),
13237            ('Comment', 'V1024'),
13238            ('ImageHistory', 'V1024'),
13239            ('Scaling', '16f4'),
13240            ('ImageStatistics', '16c16'),
13241            ('ImageType', 'i4'),
13242            ('ImageDisplaType', 'i4'),
13243            ('PixelSizeX', 'f4'),  # distance between two px in x, [nm]
13244            ('PixelSizeY', 'f4'),  # distance between two px in y, [nm]
13245            ('ImageDistanceZ', 'f4'),
13246            ('ImageDistanceE', 'f4'),
13247            ('ImageMisc', '32f4'),
13248            ('TemType', 'V160'),
13249            ('TemHighTension', 'f4'),
13250            ('TemAberrations', '32f4'),
13251            ('TemEnergy', '32f4'),
13252            ('TemMode', 'i4'),
13253            ('TemMagnification', 'f4'),
13254            ('TemMagnificationCorrection', 'f4'),
13255            ('PostMagnification', 'f4'),
13256            ('TemStageType', 'i4'),
13257            ('TemStagePosition', '5f4'),  # x, y, z, a, b
13258            ('TemImageShift', '2f4'),
13259            ('TemBeamShift', '2f4'),
13260            ('TemBeamTilt', '2f4'),
13261            ('TilingParameters', '7f4'),  # 0: tiling? 1:x 2:y 3: max x
13262            #                               4: max y 5: overlap x 6: overlap y
13263            ('TemIllumination', '3f4'),  # 0: spotsize 1: intensity
13264            ('TemShutter', 'i4'),
13265            ('TemMisc', '32f4'),
13266            ('CameraType', 'V160'),
13267            ('PhysicalPixelSizeX', 'f4'),
13268            ('PhysicalPixelSizeY', 'f4'),
13269            ('OffsetX', 'i4'),
13270            ('OffsetY', 'i4'),
13271            ('BinningX', 'i4'),
13272            ('BinningY', 'i4'),
13273            ('ExposureTime', 'f4'),
13274            ('Gain', 'f4'),
13275            ('ReadoutRate', 'f4'),
13276            ('FlatfieldDescription', 'V160'),
13277            ('Sensitivity', 'f4'),
13278            ('Dose', 'f4'),
13279            ('CamMisc', '32f4'),
13280            ('FeiMicroscopeInformation', 'V1024'),
13281            ('FeiSpecimenInformation', 'V1024'),
13282            ('Magic', 'u4'),
13283        ]
13284
13285    def MM_HEADER():
13286        # Olympus FluoView MM_Header
13287        MM_DIMENSION = [
13288            ('Name', 'a16'),
13289            ('Size', 'i4'),
13290            ('Origin', 'f8'),
13291            ('Resolution', 'f8'),
13292            ('Unit', 'a64'),
13293        ]
13294        return [
13295            ('HeaderFlag', 'i2'),
13296            ('ImageType', 'u1'),
13297            ('ImageName', 'a257'),
13298            ('OffsetData', 'u4'),
13299            ('PaletteSize', 'i4'),
13300            ('OffsetPalette0', 'u4'),
13301            ('OffsetPalette1', 'u4'),
13302            ('CommentSize', 'i4'),
13303            ('OffsetComment', 'u4'),
13304            ('Dimensions', MM_DIMENSION, 10),
13305            ('OffsetPosition', 'u4'),
13306            ('MapType', 'i2'),
13307            ('MapMin', 'f8'),
13308            ('MapMax', 'f8'),
13309            ('MinValue', 'f8'),
13310            ('MaxValue', 'f8'),
13311            ('OffsetMap', 'u4'),
13312            ('Gamma', 'f8'),
13313            ('Offset', 'f8'),
13314            ('GrayChannel', MM_DIMENSION),
13315            ('OffsetThumbnail', 'u4'),
13316            ('VoiceField', 'i4'),
13317            ('OffsetVoiceField', 'u4'),
13318        ]
13319
13320    def MM_DIMENSIONS():
13321        # map FluoView MM_Header.Dimensions to axes characters
13322        return {
13323            'X': 'X',
13324            'Y': 'Y',
13325            'Z': 'Z',
13326            'T': 'T',
13327            'CH': 'C',
13328            'WAVELENGTH': 'C',
13329            'TIME': 'T',
13330            'XY': 'R',
13331            'EVENT': 'V',
13332            'EXPOSURE': 'L',
13333        }
13334
13335    def UIC_TAGS():
13336        # map Universal Imaging Corporation MetaMorph internal tag ids to
13337        # name and type
13338        from fractions import Fraction  # delayed import
13339
13340        return [
13341            ('AutoScale', int),
13342            ('MinScale', int),
13343            ('MaxScale', int),
13344            ('SpatialCalibration', int),
13345            ('XCalibration', Fraction),
13346            ('YCalibration', Fraction),
13347            ('CalibrationUnits', str),
13348            ('Name', str),
13349            ('ThreshState', int),
13350            ('ThreshStateRed', int),
13351            ('tagid_10', None),  # undefined
13352            ('ThreshStateGreen', int),
13353            ('ThreshStateBlue', int),
13354            ('ThreshStateLo', int),
13355            ('ThreshStateHi', int),
13356            ('Zoom', int),
13357            ('CreateTime', julian_datetime),
13358            ('LastSavedTime', julian_datetime),
13359            ('currentBuffer', int),
13360            ('grayFit', None),
13361            ('grayPointCount', None),
13362            ('grayX', Fraction),
13363            ('grayY', Fraction),
13364            ('grayMin', Fraction),
13365            ('grayMax', Fraction),
13366            ('grayUnitName', str),
13367            ('StandardLUT', int),
13368            ('wavelength', int),
13369            ('StagePosition', '(%i,2,2)u4'),  # N xy positions as fract
13370            ('CameraChipOffset', '(%i,2,2)u4'),  # N xy offsets as fract
13371            ('OverlayMask', None),
13372            ('OverlayCompress', None),
13373            ('Overlay', None),
13374            ('SpecialOverlayMask', None),
13375            ('SpecialOverlayCompress', None),
13376            ('SpecialOverlay', None),
13377            ('ImageProperty', read_uic_image_property),
13378            ('StageLabel', '%ip'),  # N str
13379            ('AutoScaleLoInfo', Fraction),
13380            ('AutoScaleHiInfo', Fraction),
13381            ('AbsoluteZ', '(%i,2)u4'),  # N fractions
13382            ('AbsoluteZValid', '(%i,)u4'),  # N long
13383            ('Gamma', 'I'),  # 'I' uses offset
13384            ('GammaRed', 'I'),
13385            ('GammaGreen', 'I'),
13386            ('GammaBlue', 'I'),
13387            ('CameraBin', '2I'),
13388            ('NewLUT', int),
13389            ('ImagePropertyEx', None),
13390            ('PlaneProperty', int),
13391            ('UserLutTable', '(256,3)u1'),
13392            ('RedAutoScaleInfo', int),
13393            ('RedAutoScaleLoInfo', Fraction),
13394            ('RedAutoScaleHiInfo', Fraction),
13395            ('RedMinScaleInfo', int),
13396            ('RedMaxScaleInfo', int),
13397            ('GreenAutoScaleInfo', int),
13398            ('GreenAutoScaleLoInfo', Fraction),
13399            ('GreenAutoScaleHiInfo', Fraction),
13400            ('GreenMinScaleInfo', int),
13401            ('GreenMaxScaleInfo', int),
13402            ('BlueAutoScaleInfo', int),
13403            ('BlueAutoScaleLoInfo', Fraction),
13404            ('BlueAutoScaleHiInfo', Fraction),
13405            ('BlueMinScaleInfo', int),
13406            ('BlueMaxScaleInfo', int),
13407            # ('OverlayPlaneColor', read_uic_overlay_plane_color),
13408        ]
13409
13410    def PILATUS_HEADER():
13411        # PILATUS CBF Header Specification, Version 1.4
13412        # map key to [value_indices], type
13413        return {
13414            'Detector': ([slice(1, None)], str),
13415            'Pixel_size': ([1, 4], float),
13416            'Silicon': ([3], float),
13417            'Exposure_time': ([1], float),
13418            'Exposure_period': ([1], float),
13419            'Tau': ([1], float),
13420            'Count_cutoff': ([1], int),
13421            'Threshold_setting': ([1], float),
13422            'Gain_setting': ([1, 2], str),
13423            'N_excluded_pixels': ([1], int),
13424            'Excluded_pixels': ([1], str),
13425            'Flat_field': ([1], str),
13426            'Trim_file': ([1], str),
13427            'Image_path': ([1], str),
13428            # optional
13429            'Wavelength': ([1], float),
13430            'Energy_range': ([1, 2], float),
13431            'Detector_distance': ([1], float),
13432            'Detector_Voffset': ([1], float),
13433            'Beam_xy': ([1, 2], float),
13434            'Flux': ([1], str),
13435            'Filter_transmission': ([1], float),
13436            'Start_angle': ([1], float),
13437            'Angle_increment': ([1], float),
13438            'Detector_2theta': ([1], float),
13439            'Polarization': ([1], float),
13440            'Alpha': ([1], float),
13441            'Kappa': ([1], float),
13442            'Phi': ([1], float),
13443            'Phi_increment': ([1], float),
13444            'Chi': ([1], float),
13445            'Chi_increment': ([1], float),
13446            'Oscillation_axis': ([slice(1, None)], str),
13447            'N_oscillations': ([1], int),
13448            'Start_position': ([1], float),
13449            'Position_increment': ([1], float),
13450            'Shutter_time': ([1], float),
13451            'Omega': ([1], float),
13452            'Omega_increment': ([1], float),
13453        }
13454
13455    def ALLOCATIONGRANULARITY():
13456        # alignment for writing contiguous data to TIFF
13457        import mmap  # delayed import
13458
13459        return mmap.ALLOCATIONGRANULARITY
13460
13461    def MAXWORKERS():
13462        # half of CPU cores
13463        import multiprocessing  # delayed import
13464
13465        return max(multiprocessing.cpu_count() // 2, 1)
13466
13467    def CHUNKMODE():
13468        class CHUNKMODE(enum.IntEnum):
13469            NONE = 0
13470            PLANE = 1
13471            PAGE = 2
13472            FILE = 3
13473
13474        return CHUNKMODE
13475
13476
13477def read_tags(
13478    fh, byteorder, offsetsize, tagnames, customtags=None, maxifds=None
13479):
13480    """Read tags from chain of IFDs and return as list of dicts.
13481
13482    The file handle position must be at a valid IFD header.
13483    Does not work with NDPI.
13484
13485    """
13486    if offsetsize == 4:
13487        offsetformat = byteorder + 'I'
13488        tagnosize = 2
13489        tagnoformat = byteorder + 'H'
13490        tagsize = 12
13491        tagformat1 = byteorder + 'HH'
13492        tagformat2 = byteorder + 'I4s'
13493    elif offsetsize == 8:
13494        offsetformat = byteorder + 'Q'
13495        tagnosize = 8
13496        tagnoformat = byteorder + 'Q'
13497        tagsize = 20
13498        tagformat1 = byteorder + 'HH'
13499        tagformat2 = byteorder + 'Q8s'
13500    else:
13501        raise ValueError('invalid offset size')
13502
13503    if customtags is None:
13504        customtags = {}
13505    if maxifds is None:
13506        maxifds = 2 ** 32
13507
13508    result = []
13509    unpack = struct.unpack
13510    offset = fh.tell()
13511    while len(result) < maxifds:
13512        # loop over IFDs
13513        try:
13514            tagno = unpack(tagnoformat, fh.read(tagnosize))[0]
13515            if tagno > 4096:
13516                raise TiffFileError('suspicious number of tags')
13517        except Exception:
13518            log_warning(f'read_tags: corrupted tag list at offset {offset}')
13519            break
13520
13521        tags = {}
13522        data = fh.read(tagsize * tagno)
13523        pos = fh.tell()
13524        index = 0
13525        for _ in range(tagno):
13526            code, dtype = unpack(tagformat1, data[index : index + 4])
13527            count, value = unpack(
13528                tagformat2, data[index + 4 : index + tagsize]
13529            )
13530            index += tagsize
13531            name = tagnames.get(code, str(code))
13532            try:
13533                dataformat = TIFF.DATA_FORMATS[dtype]
13534            except KeyError:
13535                raise TiffFileError(f'unknown tag data type {dtype}')
13536
13537            fmt = '{}{}{}'.format(
13538                byteorder, count * int(dataformat[0]), dataformat[1]
13539            )
13540            size = struct.calcsize(fmt)
13541            if size > offsetsize or code in customtags:
13542                offset = unpack(offsetformat, value)[0]
13543                if offset < 8 or offset > fh.size - size:
13544                    raise TiffFileError(f'invalid tag value offset {offset}')
13545                fh.seek(offset)
13546                if code in customtags:
13547                    readfunc = customtags[code][1]
13548                    value = readfunc(fh, byteorder, dtype, count, offsetsize)
13549                elif dtype == 7 or (count > 1 and dtype == 1):
13550                    value = read_bytes(fh, byteorder, dtype, count, offsetsize)
13551                elif code in tagnames or dtype == 2:
13552                    value = unpack(fmt, fh.read(size))
13553                else:
13554                    value = read_numpy(fh, byteorder, dtype, count, offsetsize)
13555            elif dtype in (1, 7):
13556                # BYTES, UNDEFINED
13557                value = value[:size]
13558            else:
13559                value = unpack(fmt, value[:size])
13560
13561            if code not in customtags and code not in TIFF.TAG_TUPLE:
13562                if len(value) == 1:
13563                    value = value[0]
13564            if dtype == 2 and isinstance(value, bytes):
13565                # TIFF ASCII fields can contain multiple strings,
13566                #   each terminated with a NUL
13567                try:
13568                    value = bytes2str(stripnull(value, first=False).strip())
13569                except UnicodeDecodeError:
13570                    log_warning(
13571                        'read_tags: coercing invalid ASCII to bytes '
13572                        f'(tag {code})'
13573                    )
13574            tags[name] = value
13575
13576        result.append(tags)
13577        # read offset to next page
13578        fh.seek(pos)
13579        offset = unpack(offsetformat, fh.read(offsetsize))[0]
13580        if offset == 0:
13581            break
13582        if offset >= fh.size:
13583            log_warning(f'read_tags: invalid page offset ({offset})')
13584            break
13585        fh.seek(offset)
13586
13587    if result and maxifds == 1:
13588        result = result[0]
13589    return result
13590
13591
13592def read_exif_ifd(fh, byteorder, dtype, count, offsetsize):
13593    """Read EXIF tags from file and return as dict."""
13594    exif = read_tags(fh, byteorder, offsetsize, TIFF.EXIF_TAGS, maxifds=1)
13595    for name in ('ExifVersion', 'FlashpixVersion'):
13596        try:
13597            exif[name] = bytes2str(exif[name])
13598        except Exception:
13599            pass
13600    if 'UserComment' in exif:
13601        idcode = exif['UserComment'][:8]
13602        try:
13603            if idcode == b'ASCII\x00\x00\x00':
13604                exif['UserComment'] = bytes2str(exif['UserComment'][8:])
13605            elif idcode == b'UNICODE\x00':
13606                exif['UserComment'] = exif['UserComment'][8:].decode('utf-16')
13607        except Exception:
13608            pass
13609    return exif
13610
13611
13612def read_gps_ifd(fh, byteorder, dtype, count, offsetsize):
13613    """Read GPS tags from file and return as dict."""
13614    return read_tags(fh, byteorder, offsetsize, TIFF.GPS_TAGS, maxifds=1)
13615
13616
13617def read_interoperability_ifd(fh, byteorder, dtype, count, offsetsize):
13618    """Read Interoperability tags from file and return as dict."""
13619    return read_tags(fh, byteorder, offsetsize, TIFF.IOP_TAGS, maxifds=1)
13620
13621
13622def read_bytes(fh, byteorder, dtype, count, offsetsize):
13623    """Read tag data from file and return as bytes."""
13624    dtype = 'B' if dtype == 2 else byteorder + TIFF.DATA_FORMATS[dtype][-1]
13625    count *= numpy.dtype(dtype).itemsize
13626    data = fh.read(count)
13627    if len(data) != count:
13628        log_warning(
13629            f'read_bytes: failed to read all bytes ({len(data)} < {count})'
13630        )
13631    return data
13632
13633
13634def read_utf8(fh, byteorder, dtype, count, offsetsize):
13635    """Read tag data from file and return as Unicode string."""
13636    return fh.read(count).decode()
13637
13638
13639def read_numpy(fh, byteorder, dtype, count, offsetsize):
13640    """Read tag data from file and return as numpy array."""
13641    dtype = 'b' if dtype == 2 else byteorder + TIFF.DATA_FORMATS[dtype][-1]
13642    return fh.read_array(dtype, count)
13643
13644
13645def read_colormap(fh, byteorder, dtype, count, offsetsize):
13646    """Read ColorMap data from file and return as numpy array."""
13647    cmap = fh.read_array(byteorder + TIFF.DATA_FORMATS[dtype][-1], count)
13648    cmap.shape = (3, -1)
13649    return cmap
13650
13651
13652def read_json(fh, byteorder, dtype, count, offsetsize):
13653    """Read JSON tag data from file and return as object."""
13654    data = fh.read(count)
13655    try:
13656        return json.loads(stripnull(data).decode())
13657    except ValueError:
13658        log_warning('read_json: invalid JSON')
13659    return None
13660
13661
13662def read_mm_header(fh, byteorder, dtype, count, offsetsize):
13663    """Read FluoView mm_header tag from file and return as dict."""
13664    mmh = fh.read_record(TIFF.MM_HEADER, byteorder=byteorder)
13665    mmh = recarray2dict(mmh)
13666    mmh['Dimensions'] = [
13667        (bytes2str(d[0]).strip(), d[1], d[2], d[3], bytes2str(d[4]).strip())
13668        for d in mmh['Dimensions']
13669    ]
13670    d = mmh['GrayChannel']
13671    mmh['GrayChannel'] = (
13672        bytes2str(d[0]).strip(),
13673        d[1],
13674        d[2],
13675        d[3],
13676        bytes2str(d[4]).strip(),
13677    )
13678    return mmh
13679
13680
13681def read_mm_stamp(fh, byteorder, dtype, count, offsetsize):
13682    """Read FluoView mm_stamp tag from file and return as numpy.ndarray."""
13683    return fh.read_array(byteorder + 'f8', 8)
13684
13685
13686def read_uic1tag(fh, byteorder, dtype, count, offsetsize, planecount=None):
13687    """Read MetaMorph STK UIC1Tag from file and return as dict.
13688
13689    Return empty dictionary if planecount is unknown.
13690
13691    """
13692    if dtype not in (4, 5) or byteorder != '<':
13693        raise ValueError(f'invalid UIC1Tag {byteorder}{dtype}')
13694    result = {}
13695    if dtype == 5:
13696        # pre MetaMorph 2.5 (not tested)
13697        values = fh.read_array('<u4', 2 * count).reshape(count, 2)
13698        result = {'ZDistance': values[:, 0] / values[:, 1]}
13699    elif planecount:
13700        for _ in range(count):
13701            tagid = struct.unpack('<I', fh.read(4))[0]
13702            if tagid in (28, 29, 37, 40, 41):
13703                # silently skip unexpected tags
13704                fh.read(4)
13705                continue
13706            name, value = read_uic_tag(fh, tagid, planecount, offset=True)
13707            result[name] = value
13708    return result
13709
13710
13711def read_uic2tag(fh, byteorder, dtype, planecount, offsetsize):
13712    """Read MetaMorph STK UIC2Tag from file and return as dict."""
13713    if dtype != 5 or byteorder != '<':
13714        raise ValueError('invalid UIC2Tag')
13715    values = fh.read_array('<u4', 6 * planecount).reshape(planecount, 6)
13716    return {
13717        'ZDistance': values[:, 0] / values[:, 1],
13718        'DateCreated': values[:, 2],  # julian days
13719        'TimeCreated': values[:, 3],  # milliseconds
13720        'DateModified': values[:, 4],  # julian days
13721        'TimeModified': values[:, 5],  # milliseconds
13722    }
13723
13724
13725def read_uic3tag(fh, byteorder, dtype, planecount, offsetsize):
13726    """Read MetaMorph STK UIC3Tag from file and return as dict."""
13727    if dtype != 5 or byteorder != '<':
13728        raise ValueError('invalid UIC3Tag')
13729    values = fh.read_array('<u4', 2 * planecount).reshape(planecount, 2)
13730    return {'Wavelengths': values[:, 0] / values[:, 1]}
13731
13732
13733def read_uic4tag(fh, byteorder, dtype, planecount, offsetsize):
13734    """Read MetaMorph STK UIC4Tag from file and return as dict."""
13735    if dtype != 4 or byteorder != '<':
13736        raise ValueError('invalid UIC4Tag')
13737    result = {}
13738    while True:
13739        tagid = struct.unpack('<H', fh.read(2))[0]
13740        if tagid == 0:
13741            break
13742        name, value = read_uic_tag(fh, tagid, planecount, offset=False)
13743        result[name] = value
13744    return result
13745
13746
13747def read_uic_tag(fh, tagid, planecount, offset):
13748    """Read a single UIC tag value from file and return tag name and value.
13749
13750    UIC1Tags use an offset.
13751
13752    """
13753
13754    def read_int(count=1):
13755        value = struct.unpack(f'<{count}I', fh.read(4 * count))
13756        return value[0] if count == 1 else value
13757
13758    try:
13759        name, dtype = TIFF.UIC_TAGS[tagid]
13760    except IndexError:
13761        # unknown tag
13762        return f'_TagId{tagid}', read_int()
13763
13764    Fraction = TIFF.UIC_TAGS[4][1]
13765
13766    if offset:
13767        pos = fh.tell()
13768        if dtype not in (int, None):
13769            off = read_int()
13770            if off < 8:
13771                if dtype is str:
13772                    return name, ''
13773                log_warning(
13774                    f'read_uic_tag: invalid offset for tag {name!r} @{off}'
13775                )
13776                return name, off
13777            fh.seek(off)
13778
13779    if dtype is None:
13780        # skip
13781        name = '_' + name
13782        value = read_int()
13783    elif dtype is int:
13784        # int
13785        value = read_int()
13786    elif dtype is Fraction:
13787        # fraction
13788        value = read_int(2)
13789        value = value[0] / value[1]
13790    elif dtype is julian_datetime:
13791        # datetime
13792        value = julian_datetime(*read_int(2))
13793    elif dtype is read_uic_image_property:
13794        # ImagePropertyEx
13795        value = read_uic_image_property(fh)
13796    elif dtype is str:
13797        # pascal string
13798        size = read_int()
13799        if 0 <= size < 2 ** 10:
13800            value = struct.unpack(f'{size}s', fh.read(size))[0][:-1]
13801            value = bytes2str(stripnull(value))
13802        elif offset:
13803            value = ''
13804            log_warning(f'read_uic_tag: corrupt string in tag {name!r}')
13805        else:
13806            raise ValueError(f'read_uic_tag: invalid string size {size}')
13807    elif dtype == '%ip':
13808        # sequence of pascal strings
13809        value = []
13810        for _ in range(planecount):
13811            size = read_int()
13812            if 0 <= size < 2 ** 10:
13813                string = struct.unpack(f'{size}s', fh.read(size))[0][:-1]
13814                string = bytes2str(stripnull(string))
13815                value.append(string)
13816            elif offset:
13817                log_warning(f'read_uic_tag: corrupt string in tag {name!r}')
13818            else:
13819                raise ValueError(f'read_uic_tag: invalid string size: {size}')
13820    else:
13821        # struct or numpy type
13822        dtype = '<' + dtype
13823        if '%i' in dtype:
13824            dtype = dtype % planecount
13825        if '(' in dtype:
13826            # numpy type
13827            value = fh.read_array(dtype, 1)[0]
13828            if value.shape[-1] == 2:
13829                # assume fractions
13830                value = value[..., 0] / value[..., 1]
13831        else:
13832            # struct format
13833            value = struct.unpack(dtype, fh.read(struct.calcsize(dtype)))
13834            if len(value) == 1:
13835                value = value[0]
13836
13837    if offset:
13838        fh.seek(pos + 4)
13839
13840    return name, value
13841
13842
13843def read_uic_image_property(fh):
13844    """Read UIC ImagePropertyEx tag from file and return as dict."""
13845    # TODO: test this
13846    size = struct.unpack('B', fh.read(1))[0]
13847    name = struct.unpack(f'{size}s', fh.read(size))[0][:-1]
13848    flags, prop = struct.unpack('<IB', fh.read(5))
13849    if prop == 1:
13850        value = struct.unpack('II', fh.read(8))
13851        value = value[0] / value[1]
13852    else:
13853        size = struct.unpack('B', fh.read(1))[0]
13854        value = struct.unpack(f'{size}s', fh.read(size))[0]
13855    return dict(name=name, flags=flags, value=value)
13856
13857
13858def read_cz_lsminfo(fh, byteorder, dtype, count, offsetsize):
13859    """Read CZ_LSMINFO tag from file and return as dict."""
13860    if byteorder != '<':
13861        raise ValueError('invalid CZ_LSMINFO structure')
13862    magic_number, structure_size = struct.unpack('<II', fh.read(8))
13863    if magic_number not in (50350412, 67127628):
13864        raise ValueError('invalid CZ_LSMINFO structure')
13865    fh.seek(-8, os.SEEK_CUR)
13866
13867    if structure_size < numpy.dtype(TIFF.CZ_LSMINFO).itemsize:
13868        # adjust structure according to structure_size
13869        lsminfo = []
13870        size = 0
13871        for name, dtype in TIFF.CZ_LSMINFO:
13872            size += numpy.dtype(dtype).itemsize
13873            if size > structure_size:
13874                break
13875            lsminfo.append((name, dtype))
13876    else:
13877        lsminfo = TIFF.CZ_LSMINFO
13878
13879    lsminfo = fh.read_record(lsminfo, byteorder=byteorder)
13880    lsminfo = recarray2dict(lsminfo)
13881
13882    # read LSM info subrecords at offsets
13883    for name, reader in TIFF.CZ_LSMINFO_READERS.items():
13884        if reader is None:
13885            continue
13886        offset = lsminfo.get('Offset' + name, 0)
13887        if offset < 8:
13888            continue
13889        fh.seek(offset)
13890        try:
13891            lsminfo[name] = reader(fh)
13892        except ValueError:
13893            pass
13894    return lsminfo
13895
13896
13897def read_lsm_channeldatatypes(fh):
13898    """Read LSM channel data type."""
13899    size = struct.unpack('<I', fh.read(4))[0]
13900    return fh.read_array('<u4', count=size)
13901
13902
13903def read_lsm_channelwavelength(fh):
13904    """Read LSM channel wavelength ranges from file and return as list."""
13905    size = struct.unpack('<i', fh.read(4))[0]
13906    return fh.read_array('<2f8', count=size)
13907
13908
13909def read_lsm_positions(fh):
13910    """Read LSM positions from file and return as list."""
13911    size = struct.unpack('<I', fh.read(4))[0]
13912    return fh.read_array('<3f8', count=size)
13913
13914
13915def read_lsm_timestamps(fh):
13916    """Read LSM time stamps from file and return as list."""
13917    size, count = struct.unpack('<ii', fh.read(8))
13918    if size != (8 + 8 * count):
13919        log_warning('read_lsm_timestamps: invalid LSM TimeStamps block')
13920        return []
13921    # return struct.unpack(f'<{count}d', fh.read(8 * count))
13922    return fh.read_array('<f8', count=count)
13923
13924
13925def read_lsm_eventlist(fh):
13926    """Read LSM events from file and return as list of (time, type, text)."""
13927    count = struct.unpack('<II', fh.read(8))[1]
13928    events = []
13929    while count > 0:
13930        esize, etime, etype = struct.unpack('<IdI', fh.read(16))
13931        etext = bytes2str(stripnull(fh.read(esize - 16)))
13932        events.append((etime, etype, etext))
13933        count -= 1
13934    return events
13935
13936
13937def read_lsm_channelcolors(fh):
13938    """Read LSM ChannelColors structure from file and return as dict."""
13939    result = {'Mono': False, 'Colors': [], 'ColorNames': []}
13940    pos = fh.tell()
13941    (size, ncolors, nnames, coffset, noffset, mono) = struct.unpack(
13942        '<IIIIII', fh.read(24)
13943    )
13944    if ncolors != nnames:
13945        log_warning(
13946            'read_lsm_channelcolors: invalid LSM ChannelColors structure'
13947        )
13948        return result
13949    result['Mono'] = bool(mono)
13950    # Colors
13951    fh.seek(pos + coffset)
13952    colors = fh.read_array('uint8', count=ncolors * 4).reshape((ncolors, 4))
13953    result['Colors'] = colors.tolist()
13954    # ColorNames
13955    fh.seek(pos + noffset)
13956    buffer = fh.read(size - noffset)
13957    names = []
13958    while len(buffer) > 4:
13959        size = struct.unpack('<I', buffer[:4])[0]
13960        names.append(bytes2str(buffer[4 : 3 + size]))
13961        buffer = buffer[4 + size :]
13962    result['ColorNames'] = names
13963    return result
13964
13965
13966def read_lsm_lookuptable(fh):
13967    """Read LSM lookup tables from file and return as dict."""
13968    result = {}
13969    (
13970        size,
13971        nsubblocks,
13972        nchannels,
13973        luttype,
13974        advanced,
13975        currentchannel,
13976    ) = struct.unpack('<iiiiii', fh.read(24))
13977    if size < 60:
13978        log_warning('read_lsm_lookuptable: invalid LSM LookupTables structure')
13979        return result
13980    fh.read(9 * 4)  # reserved
13981    result['LutType'] = TIFF.CZ_LSM_LUTTYPE(luttype)
13982    result['Advanced'] = advanced
13983    result['NumberChannels'] = nchannels
13984    result['CurrentChannel'] = currentchannel
13985    result['SubBlocks'] = subblocks = []
13986    for i in range(nsubblocks):
13987        sbtype = struct.unpack('<i', fh.read(4))[0]
13988        if sbtype <= 0:
13989            break
13990        size = struct.unpack('<i', fh.read(4))[0] - 8
13991        if sbtype == 1:
13992            data = fh.read_array('<f8', count=nchannels)
13993        elif sbtype == 2:
13994            data = fh.read_array('<f8', count=nchannels)
13995        elif sbtype == 3:
13996            data = fh.read_array('<f8', count=nchannels)
13997        elif sbtype == 4:
13998            # the data type is wrongly documented as f8
13999            data = fh.read_array('<i4', count=nchannels * 4)
14000            data = data.reshape((-1, 2, 2))
14001        elif sbtype == 5:
14002            # the data type is wrongly documented as f8
14003            nknots = struct.unpack('<i', fh.read(4))[0]  # undocumented
14004            data = fh.read_array('<i4', count=nchannels * nknots * 2)
14005            data = data.reshape((nchannels, nknots, 2))
14006        elif sbtype == 6:
14007            data = fh.read_array('<i2', count=nchannels * 4096)
14008            data = data.reshape((-1, 4096))
14009        else:
14010            log_warning(
14011                f'read_lsm_lookuptable: invalid LSM SubBlock type {sbtype}'
14012            )
14013            break
14014        subblocks.append(
14015            {'Type': TIFF.CZ_LSM_SUBBLOCK_TYPE(sbtype), 'Data': data}
14016        )
14017    return result
14018
14019
14020def read_lsm_scaninfo(fh):
14021    """Read LSM ScanInfo structure from file and return as dict."""
14022    block = {}
14023    blocks = [block]
14024    unpack = struct.unpack
14025    if struct.unpack('<I', fh.read(4))[0] != 0x10000000:
14026        # not a Recording sub block
14027        log_warning('read_lsm_scaninfo: invalid LSM ScanInfo structure')
14028        return block
14029    fh.read(8)
14030    while True:
14031        entry, dtype, size = unpack('<III', fh.read(12))
14032        if dtype == 2:
14033            # ascii
14034            value = bytes2str(stripnull(fh.read(size)))
14035        elif dtype == 4:
14036            # long
14037            value = unpack('<i', fh.read(4))[0]
14038        elif dtype == 5:
14039            # rational
14040            value = unpack('<d', fh.read(8))[0]
14041        else:
14042            value = 0
14043        if entry in TIFF.CZ_LSMINFO_SCANINFO_ARRAYS:
14044            blocks.append(block)
14045            name = TIFF.CZ_LSMINFO_SCANINFO_ARRAYS[entry]
14046            newobj = []
14047            block[name] = newobj
14048            block = newobj
14049        elif entry in TIFF.CZ_LSMINFO_SCANINFO_STRUCTS:
14050            blocks.append(block)
14051            newobj = {}
14052            block.append(newobj)
14053            block = newobj
14054        elif entry in TIFF.CZ_LSMINFO_SCANINFO_ATTRIBUTES:
14055            name = TIFF.CZ_LSMINFO_SCANINFO_ATTRIBUTES[entry]
14056            block[name] = value
14057        elif entry == 0xFFFFFFFF:
14058            # end sub block
14059            block = blocks.pop()
14060        else:
14061            # unknown entry
14062            block[f'Entry0x{entry:x}'] = value
14063        if not blocks:
14064            break
14065    return block
14066
14067
14068def read_sis(fh, byteorder, dtype, count, offsetsize):
14069    """Read OlympusSIS structure and return as dict.
14070
14071    No specification is avaliable. Only few fields are known.
14072
14073    """
14074    result = {}
14075
14076    (magic, minute, hour, day, month, year, name, tagcount) = struct.unpack(
14077        '<4s6xhhhhh6x32sh', fh.read(60)
14078    )
14079
14080    if magic != b'SIS0':
14081        raise ValueError('invalid OlympusSIS structure')
14082
14083    result['name'] = bytes2str(stripnull(name))
14084    try:
14085        result['datetime'] = datetime.datetime(
14086            1900 + year, month + 1, day, hour, minute
14087        )
14088    except ValueError:
14089        pass
14090
14091    data = fh.read(8 * tagcount)
14092    for i in range(0, tagcount * 8, 8):
14093        tagtype, count, offset = struct.unpack('<hhI', data[i : i + 8])
14094        fh.seek(offset)
14095        if tagtype == 1:
14096            # general data
14097            (lenexp, xcal, ycal, mag, camname, pictype) = struct.unpack(
14098                '<10xhdd8xd2x34s32s', fh.read(112)  # 220
14099            )
14100            m = math.pow(10, lenexp)
14101            result['pixelsizex'] = xcal * m
14102            result['pixelsizey'] = ycal * m
14103            result['magnification'] = mag
14104            result['cameraname'] = bytes2str(stripnull(camname))
14105            result['picturetype'] = bytes2str(stripnull(pictype))
14106        elif tagtype == 10:
14107            # channel data
14108            continue
14109            # TODO: does not seem to work?
14110            # (length, _, exptime, emv, _, camname, _, mictype,
14111            #  ) = struct.unpack('<h22sId4s32s48s32s', fh.read(152))  # 720
14112            # result['exposuretime'] = exptime
14113            # result['emvoltage'] = emv
14114            # result['cameraname2'] = bytes2str(stripnull(camname))
14115            # result['microscopename'] = bytes2str(stripnull(mictype))
14116
14117    return result
14118
14119
14120def read_sis_ini(fh, byteorder, dtype, count, offsetsize):
14121    """Read OlympusSIS INI string and return as dict."""
14122    inistr = fh.read(count)
14123    inistr = bytes2str(stripnull(inistr))
14124    try:
14125        return olympusini_metadata(inistr)
14126    except Exception as exc:
14127        log_warning(f'olympusini_metadata: {exc.__class__.__name__}: {exc}')
14128        return {}
14129
14130
14131def read_tvips_header(fh, byteorder, dtype, count, offsetsize):
14132    """Read TVIPS EM-MENU headers and return as dict."""
14133    result = {}
14134    header = fh.read_record(TIFF.TVIPS_HEADER_V1, byteorder=byteorder)
14135    for name, typestr in TIFF.TVIPS_HEADER_V1:
14136        result[name] = header[name].tolist()
14137    if header['Version'] == 2:
14138        header = fh.read_record(TIFF.TVIPS_HEADER_V2, byteorder=byteorder)
14139        if header['Magic'] != int(0xAAAAAAAA):
14140            log_warning('read_tvips_header: invalid TVIPS v2 magic number')
14141            return {}
14142        # decode utf16 strings
14143        for name, typestr in TIFF.TVIPS_HEADER_V2:
14144            if typestr.startswith('V'):
14145                s = header[name].tobytes().decode('utf-16', errors='ignore')
14146                result[name] = stripnull(s, null='\x00')
14147            else:
14148                result[name] = header[name].tolist()
14149        # convert nm to m
14150        for axis in 'XY':
14151            header['PhysicalPixelSize' + axis] /= 1e9
14152            header['PixelSize' + axis] /= 1e9
14153    elif header.version != 1:
14154        log_warning('read_tvips_header: unknown TVIPS header version')
14155        return {}
14156    return result
14157
14158
14159def read_fei_metadata(fh, byteorder, dtype, count, offsetsize):
14160    """Read FEI SFEG/HELIOS headers and return as dict."""
14161    result = {}
14162    section = {}
14163    data = bytes2str(stripnull(fh.read(count)))
14164    for line in data.splitlines():
14165        line = line.strip()
14166        if line.startswith('['):
14167            section = {}
14168            result[line[1:-1]] = section
14169            continue
14170        try:
14171            key, value = line.split('=')
14172        except ValueError:
14173            continue
14174        section[key] = astype(value)
14175    return result
14176
14177
14178def read_cz_sem(fh, byteorder, dtype, count, offsetsize):
14179    """Read Zeiss SEM tag and return as dict.
14180
14181    See https://sourceforge.net/p/gwyddion/mailman/message/29275000/ for
14182    unnamed values.
14183
14184    """
14185    result = {'': ()}
14186    key = None
14187    data = bytes2str(stripnull(fh.read(count)))
14188    for line in data.splitlines():
14189        if line.isupper():
14190            key = line.lower()
14191        elif key:
14192            try:
14193                name, value = line.split('=')
14194            except ValueError:
14195                try:
14196                    name, value = line.split(':', 1)
14197                except Exception:
14198                    continue
14199            value = value.strip()
14200            unit = ''
14201            try:
14202                v, u = value.split()
14203                number = astype(v, (int, float))
14204                if number != v:
14205                    value = number
14206                    unit = u
14207            except Exception:
14208                number = astype(value, (int, float))
14209                if number != value:
14210                    value = number
14211                if value in ('No', 'Off'):
14212                    value = False
14213                elif value in ('Yes', 'On'):
14214                    value = True
14215            result[key] = (name.strip(), value)
14216            if unit:
14217                result[key] += (unit,)
14218            key = None
14219        else:
14220            result[''] += (astype(line, (int, float)),)
14221    return result
14222
14223
14224def read_nih_image_header(fh, byteorder, dtype, count, offsetsize):
14225    """Read NIH_IMAGE_HEADER tag from file and return as dict."""
14226    a = fh.read_record(TIFF.NIH_IMAGE_HEADER, byteorder=byteorder)
14227    a = a.newbyteorder(byteorder)
14228    a = recarray2dict(a)
14229    a['XUnit'] = a['XUnit'][: a['XUnitSize']]
14230    a['UM'] = a['UM'][: a['UMsize']]
14231    return a
14232
14233
14234def read_scanimage_metadata(fh):
14235    """Read ScanImage BigTIFF v3 or v4 static and ROI metadata from open file.
14236
14237    Return non-varying frame data, ROI group data, and version as
14238    tuple(dict, dict, int).
14239
14240    The settings can be used to read image data and metadata without parsing
14241    the TIFF file.
14242
14243    Raise ValueError if file does not contain valid ScanImage metadata.
14244
14245    Frame data and ROI groups can alternatively be obtained from the Software
14246    and Artist tags of any TIFF page.
14247
14248    """
14249    fh.seek(0)
14250    try:
14251        byteorder, version = struct.unpack('<2sH', fh.read(4))
14252        if byteorder != b'II' or version != 43:
14253            raise ValueError('not a BigTIFF file')
14254        fh.seek(16)
14255        magic, version, size0, size1 = struct.unpack('<IIII', fh.read(16))
14256        if magic != 117637889 or version not in (3, 4):
14257            raise ValueError(
14258                f'invalid magic {magic} or version {version} number'
14259            )
14260    except UnicodeDecodeError as exc:
14261        raise ValueError('file must be opened in binary mode') from exc
14262    except Exception as exc:
14263        raise ValueError('not a ScanImage BigTIFF v3 or v4 file') from exc
14264
14265    frame_data = matlabstr2py(bytes2str(fh.read(size0)[:-1]))
14266    roi_data = read_json(fh, '<', None, size1, None) if size1 > 1 else {}
14267    return frame_data, roi_data, version
14268
14269
14270def read_micromanager_metadata(fh):
14271    """Read MicroManager non-TIFF settings from open file and return as dict.
14272
14273    The settings can be used to read image data without parsing the TIFF file.
14274
14275    """
14276    fh.seek(0)
14277    try:
14278        byteorder = {b'II': '<', b'MM': '>'}[fh.read(2)]
14279    except IndexError:
14280        raise ValueError('not a MicroManager TIFF file')
14281
14282    result = {}
14283    fh.seek(8)
14284    (
14285        index_header,  # Index map
14286        index_offset,
14287        display_header,
14288        display_offset,
14289        comments_header,
14290        comments_offset,
14291        summary_header,
14292        summary_length,
14293    ) = struct.unpack(byteorder + 'IIIIIIII', fh.read(32))
14294
14295    if summary_header == 2355492:
14296        result['Summary'] = read_json(
14297            fh, byteorder, None, summary_length, None
14298        )
14299    else:
14300        log_warning('invalid MicroManager summary header')
14301
14302    if index_header == 54773648:
14303        fh.seek(index_offset)
14304        header, count = struct.unpack(byteorder + 'II', fh.read(8))
14305        if header == 3453623:
14306            data = struct.unpack(
14307                byteorder + 'IIIII' * count, fh.read(20 * count)
14308            )
14309            result['IndexMap'] = {
14310                'Channel': data[::5],
14311                'Slice': data[1::5],
14312                'Frame': data[2::5],
14313                'Position': data[3::5],
14314                'Offset': data[4::5],
14315            }
14316        else:
14317            log_warning('invalid MicroManager index header')
14318    else:
14319        log_warning('invalid MicroManager index header')
14320
14321    if display_header == 483765892:
14322        fh.seek(display_offset)
14323        header, count = struct.unpack(byteorder + 'II', fh.read(8))
14324        if header == 347834724:
14325            result['DisplaySettings'] = read_json(
14326                fh, byteorder, None, count, None
14327            )
14328        else:
14329            log_warning('invalid MicroManager display header')
14330    else:
14331        log_warning('invalid MicroManager display header')
14332
14333    result['MajorVersion'] = 0
14334    if comments_header == 99384722:
14335        # Micro-Manager multipage TIFF
14336        fh.seek(comments_offset)
14337        header, count = struct.unpack(byteorder + 'II', fh.read(8))
14338        if header == 84720485:
14339            result['Comments'] = read_json(fh, byteorder, None, count, None)
14340        else:
14341            log_warning('invalid MicroManager comments header')
14342    elif comments_header == 483729:
14343        # NDTiffStorage
14344        result['MajorVersion'] = comments_offset
14345    else:
14346        log_warning('invalid MicroManager comments header')
14347
14348    return result
14349
14350
14351def read_metaseries_catalog(fh):
14352    """Read MetaSeries non-TIFF hint catalog from file.
14353
14354    Raise ValueError if the file does not contain a valid hint catalog.
14355
14356    """
14357    # TODO: implement read_metaseries_catalog
14358    raise NotImplementedError
14359
14360
14361def imagej_metadata_tag(metadata, byteorder):
14362    """Return IJMetadata and IJMetadataByteCounts tags from metadata dict.
14363
14364    The tags can be passed to TiffWriter.write() as extratags.
14365
14366    The metadata dict may contain the following keys and values:
14367
14368        Info : str
14369            Human-readable information as string.
14370        Labels : sequence of str
14371            Human-readable labels for each channel.
14372        Ranges : sequence of doubles
14373            Lower and upper values for each channel.
14374        LUTs : sequence of (3, 256) uint8 ndarrays
14375            Color palettes for each channel.
14376        Plot : bytes
14377            Undocumented ImageJ internal format.
14378        ROI: bytes
14379            Undocumented ImageJ internal region of interest format.
14380        Overlays : bytes
14381            Undocumented ImageJ internal format.
14382
14383    """
14384    if not metadata:
14385        return ()
14386    header = [{'>': b'IJIJ', '<': b'JIJI'}[byteorder]]
14387    bytecounts = [0]
14388    body = []
14389
14390    def _string(data, byteorder):
14391        return data.encode('utf-16' + {'>': 'be', '<': 'le'}[byteorder])
14392
14393    def _doubles(data, byteorder):
14394        return struct.pack(byteorder + ('d' * len(data)), *data)
14395
14396    def _ndarray(data, byteorder):
14397        return data.tobytes()
14398
14399    def _bytes(data, byteorder):
14400        return data
14401
14402    metadata_types = (
14403        ('Info', b'info', _string),
14404        ('Labels', b'labl', _string),
14405        ('Ranges', b'rang', _doubles),
14406        ('LUTs', b'luts', _ndarray),
14407        ('Plot', b'plot', _bytes),
14408        ('ROI', b'roi ', _bytes),
14409        ('Overlays', b'over', _bytes),
14410    )
14411
14412    for key, mtype, func in metadata_types:
14413        if key.lower() in metadata:
14414            key = key.lower()
14415        elif key not in metadata:
14416            continue
14417        if byteorder == '<':
14418            mtype = mtype[::-1]
14419        values = metadata[key]
14420        if isinstance(values, list):
14421            count = len(values)
14422        else:
14423            values = [values]
14424            count = 1
14425        header.append(mtype + struct.pack(byteorder + 'I', count))
14426        for value in values:
14427            data = func(value, byteorder)
14428            body.append(data)
14429            bytecounts.append(len(data))
14430
14431    if not body:
14432        return ()
14433    body = b''.join(body)
14434    header = b''.join(header)
14435    data = header + body
14436    bytecounts[0] = len(header)
14437    bytecounts = struct.pack(byteorder + ('I' * len(bytecounts)), *bytecounts)
14438    return (
14439        (50839, 1, len(data), data, True),
14440        (50838, 4, len(bytecounts) // 4, bytecounts, True),
14441    )
14442
14443
14444def imagej_metadata(data, bytecounts, byteorder):
14445    """Return IJMetadata tag value as dict.
14446
14447    The 'Info' string can have multiple formats, e.g. OIF or ScanImage,
14448    that might be parsed into dicts using the matlabstr2py or
14449    oiffile.SettingsFile functions.
14450    'ROI' and 'Overlays' are returned as bytes, which can be parsed with the
14451    ImagejRoi.frombytes() function of the roifile package.
14452
14453    """
14454
14455    def _string(data, byteorder):
14456        return data.decode('utf-16' + {'>': 'be', '<': 'le'}[byteorder])
14457
14458    def _doubles(data, byteorder):
14459        return struct.unpack(byteorder + ('d' * (len(data) // 8)), data)
14460
14461    def _lut(data, byteorder):
14462        return numpy.frombuffer(data, 'uint8').reshape(-1, 256)
14463
14464    def _bytes(data, byteorder):
14465        return data
14466
14467    # big-endian
14468    metadata_types = {
14469        b'info': ('Info', _string),
14470        b'labl': ('Labels', _string),
14471        b'rang': ('Ranges', _doubles),
14472        b'luts': ('LUTs', _lut),
14473        b'plot': ('Plot', _bytes),
14474        b'roi ': ('ROI', _bytes),
14475        b'over': ('Overlays', _bytes),
14476    }
14477    # little-endian
14478    metadata_types.update({k[::-1]: v for k, v in metadata_types.items()})
14479
14480    if not bytecounts:
14481        raise ValueError('no ImageJ metadata')
14482
14483    if not data[:4] in (b'IJIJ', b'JIJI'):
14484        raise ValueError('invalid ImageJ metadata')
14485
14486    header_size = bytecounts[0]
14487    if header_size < 12 or header_size > 804:
14488        raise ValueError('invalid ImageJ metadata header size')
14489
14490    ntypes = (header_size - 4) // 8
14491    header = struct.unpack(
14492        byteorder + '4sI' * ntypes, data[4 : 4 + ntypes * 8]
14493    )
14494    pos = 4 + ntypes * 8
14495    counter = 0
14496    result = {}
14497    for mtype, count in zip(header[::2], header[1::2]):
14498        values = []
14499        name, func = metadata_types.get(mtype, (bytes2str(mtype), read_bytes))
14500        for _ in range(count):
14501            counter += 1
14502            pos1 = pos + bytecounts[counter]
14503            values.append(func(data[pos:pos1], byteorder))
14504            pos = pos1
14505        result[name.strip()] = values[0] if count == 1 else values
14506    return result
14507
14508
14509def imagej_description_metadata(description):
14510    r"""Return metatata from ImageJ image description as dict.
14511
14512    Raise ValueError if not a valid ImageJ description.
14513
14514    >>> description = 'ImageJ=1.11a\nimages=510\nhyperstack=true\n'
14515    >>> imagej_description_metadata(description)  # doctest: +SKIP
14516    {'ImageJ': '1.11a', 'images': 510, 'hyperstack': True}
14517
14518    """
14519
14520    def _bool(val):
14521        return {'true': True, 'false': False}[val.lower()]
14522
14523    result = {}
14524    for line in description.splitlines():
14525        try:
14526            key, val = line.split('=')
14527        except Exception:
14528            continue
14529        key = key.strip()
14530        val = val.strip()
14531        for dtype in (int, float, _bool):
14532            try:
14533                val = dtype(val)
14534                break
14535            except Exception:
14536                pass
14537        result[key] = val
14538
14539    if 'ImageJ' not in result:
14540        raise ValueError('not an ImageJ image description')
14541    return result
14542
14543
14544def imagej_description(
14545    shape,
14546    rgb=None,
14547    colormaped=False,
14548    version=None,
14549    hyperstack=None,
14550    mode=None,
14551    loop=None,
14552    **kwargs,
14553):
14554    """Return ImageJ image description from data shape.
14555
14556    ImageJ can handle up to 6 dimensions in order TZCYXS.
14557
14558    >>> imagej_description((51, 5, 2, 196, 171))  # doctest: +SKIP
14559    ImageJ=1.11a
14560    images=510
14561    channels=2
14562    slices=5
14563    frames=51
14564    hyperstack=true
14565    mode=grayscale
14566    loop=false
14567
14568    """
14569    if colormaped:
14570        raise NotImplementedError('ImageJ colormapping not supported')
14571    if version is None:
14572        version = kwargs.pop('ImageJ', '1.11a')
14573    axes = kwargs.pop('axes', None)
14574    shape = imagej_shape(shape, rgb=rgb, axes=axes)
14575    rgb = shape[-1] in (3, 4)
14576
14577    append = []
14578    result = [f'ImageJ={version}']
14579    result.append(f'images={product(shape[:-3])}')
14580    if hyperstack is None:
14581        hyperstack = True
14582        append.append('hyperstack=true')
14583    else:
14584        append.append(f'hyperstack={bool(hyperstack)}')
14585    if shape[2] > 1:
14586        result.append(f'channels={shape[2]}')
14587    if mode is None and not rgb:
14588        mode = 'grayscale'
14589    if hyperstack and mode:
14590        append.append(f'mode={mode}')
14591    if shape[1] > 1:
14592        result.append(f'slices={shape[1]}')
14593    if shape[0] > 1:
14594        result.append(f'frames={shape[0]}')
14595        if loop is None:
14596            append.append('loop=false')
14597    if loop is not None:
14598        append.append(f'loop={bool(loop)}'.lower())
14599
14600    for key, value in kwargs.items():
14601        if key not in ('images', 'channels', 'slices', 'frames'):
14602            append.append(f'{key.lower()}={value}')
14603
14604    return '\n'.join(result + append + [''])
14605
14606
14607def imagej_shape(shape, rgb=None, axes=None):
14608    """Return shape normalized to 6D ImageJ hyperstack TZCYXS.
14609
14610    Raise ValueError if not a valid ImageJ hyperstack shape or axes order.
14611
14612    >>> imagej_shape((2, 3, 4, 5, 3), False)
14613    (2, 3, 4, 5, 3, 1)
14614
14615    """
14616    shape = tuple(int(i) for i in shape)
14617    ndim = len(shape)
14618    if 1 > ndim > 6:
14619        raise ValueError('ImageJ hyperstack must be 2-6 dimensional')
14620
14621    if axes:
14622        if len(axes) != ndim:
14623            raise ValueError('ImageJ hyperstack shape and axes do not match')
14624        i = 0
14625        axes = axes.upper()
14626        for ax in axes:
14627            j = 'TZCYXS'.find(ax)
14628            if j < i:
14629                raise ValueError(
14630                    'ImageJ hyperstack axes must be in TZCYXS order'
14631                )
14632            i = j
14633        ndims = len(axes)
14634        newshape = []
14635        i = 0
14636        for ax in 'TZCYXS':
14637            if i < ndims and ax == axes[i]:
14638                newshape.append(shape[i])
14639                i += 1
14640            else:
14641                newshape.append(1)
14642        if newshape[-1] not in (1, 3, 4):
14643            raise ValueError(
14644                'ImageJ hyperstack must contain 1, 3, or 4 samples'
14645            )
14646        return tuple(newshape)
14647
14648    if rgb is None:
14649        rgb = shape[-1] in (3, 4) and ndim > 2
14650    if rgb and shape[-1] not in (3, 4):
14651        raise ValueError('ImageJ hyperstack is not a RGB image')
14652    if not rgb and ndim == 6 and shape[-1] != 1:
14653        raise ValueError('ImageJ hyperstack is not a grayscale image')
14654    if rgb or shape[-1] == 1:
14655        return (1,) * (6 - ndim) + shape
14656    return (1,) * (5 - ndim) + shape + (1,)
14657
14658
14659def jpeg_decode_colorspace(photometric, planarconfig, extrasamples):
14660    """Return JPEG and output colorspace for jpeg_decode function."""
14661    colorspace = None
14662    outcolorspace = None
14663    if extrasamples:
14664        pass
14665    elif photometric == 6:
14666        # YCBCR -> RGB
14667        outcolorspace = 2  # RGB
14668    elif photometric == 2:
14669        if planarconfig == 1:
14670            colorspace = outcolorspace = 2  # RGB
14671    elif photometric == 5:
14672        # CMYK
14673        outcolorspace = 4
14674    elif photometric > 3:
14675        outcolorspace = TIFF.PHOTOMETRIC(photometric).name
14676    return colorspace, outcolorspace
14677
14678
14679def jpeg_shape(jpeg):
14680    """Return bitdepth and shape of JPEG image."""
14681    i = 0
14682    while True and i < len(jpeg):
14683        marker = struct.unpack('>H', jpeg[i : i + 2])[0]
14684        i += 2
14685
14686        if marker == 0xFFD8:
14687            # start of image
14688            continue
14689        if marker == 0xFFD9:
14690            # end of image
14691            break
14692        if 0xFFD0 <= marker <= 0xFFD7:
14693            # restart marker
14694            continue
14695        if marker == 0xFF01:
14696            # private marker
14697            continue
14698
14699        length = struct.unpack('>H', jpeg[i : i + 2])[0]
14700        i += 2
14701
14702        if 0xFFC0 <= marker <= 0xFFC3:
14703            # start of frame
14704            return struct.unpack('>BHHB', jpeg[i : i + 6])
14705        if marker == 0xFFDA:
14706            # start of scan
14707            break
14708
14709        # skip to next marker
14710        i += length - 2
14711
14712    raise ValueError('no SOF marker found')
14713
14714
14715def ndpi_jpeg_tile(jpeg):
14716    """Return tile shape and JPEG header from JPEG with restart markers."""
14717    restartinterval = 0
14718    sofoffset = 0
14719    sosoffset = 0
14720    i = 0
14721    while True and i < len(jpeg):
14722        marker = struct.unpack('>H', jpeg[i : i + 2])[0]
14723        i += 2
14724
14725        if marker == 0xFFD8:
14726            # start of image
14727            continue
14728        if marker == 0xFFD9:
14729            # end of image
14730            break
14731        if 0xFFD0 <= marker <= 0xFFD7:
14732            # restart marker
14733            continue
14734        if marker == 0xFF01:
14735            # private marker
14736            continue
14737
14738        length = struct.unpack('>H', jpeg[i : i + 2])[0]
14739        i += 2
14740
14741        if marker == 0xFFDD:
14742            # define restart interval
14743            restartinterval = struct.unpack('>H', jpeg[i : i + 2])[0]
14744
14745        elif marker == 0xFFC0:
14746            # start of frame
14747            sofoffset = i + 1
14748            precision, imlength, imwidth, ncomponents = struct.unpack(
14749                '>BHHB', jpeg[i : i + 6]
14750            )
14751            i += 6
14752            mcuwidth = 1
14753            mcuheight = 1
14754            for _ in range(ncomponents):
14755                cid, factor, table = struct.unpack('>BBB', jpeg[i : i + 3])
14756                i += 3
14757                if factor >> 4 > mcuwidth:
14758                    mcuwidth = factor >> 4
14759                if factor & 0b00001111 > mcuheight:
14760                    mcuheight = factor & 0b00001111
14761            mcuwidth *= 8
14762            mcuheight *= 8
14763            i = sofoffset - 1
14764
14765        elif marker == 0xFFDA:
14766            # start of scan
14767            sosoffset = i + length - 2
14768            break
14769
14770        # skip to next marker
14771        i += length - 2
14772
14773    if restartinterval == 0 or sofoffset == 0 or sosoffset == 0:
14774        raise ValueError('missing required JPEG markers')
14775
14776    # patch jpeg header for tile size
14777    tilelength = mcuheight
14778    tilewidth = restartinterval * mcuwidth
14779    jpegheader = (
14780        jpeg[:sofoffset]
14781        + struct.pack('>HH', tilelength, tilewidth)
14782        + jpeg[sofoffset + 4 : sosoffset]
14783    )
14784    return tilelength, tilewidth, jpegheader
14785
14786
14787def json_description(shape, **metadata):
14788    """Return JSON image description from data shape and other metadata.
14789
14790    Return UTF-8 encoded JSON.
14791
14792    >>> json_description((256, 256, 3), axes='YXS')  # doctest: +SKIP
14793    b'{"shape": [256, 256, 3], "axes": "YXS"}'
14794
14795    """
14796    metadata.update(shape=shape)
14797    return json.dumps(metadata)  # .encode()
14798
14799
14800def json_description_metadata(description):
14801    """Return metatata from JSON formated image description as dict.
14802
14803    Raise ValuError if description is of unknown format.
14804
14805    >>> description = '{"shape": [256, 256, 3], "axes": "YXS"}'
14806    >>> json_description_metadata(description)  # doctest: +SKIP
14807    {'shape': [256, 256, 3], 'axes': 'YXS'}
14808    >>> json_description_metadata('shape=(256, 256, 3)')
14809    {'shape': (256, 256, 3)}
14810
14811    """
14812    if description[:6] == 'shape=':
14813        # old-style 'shaped' description; not JSON
14814        shape = tuple(int(i) for i in description[7:-1].split(','))
14815        return dict(shape=shape)
14816    if description[:1] == '{' and description[-1:] == '}':
14817        # JSON description
14818        return json.loads(description)
14819    raise ValueError('invalid JSON image description', description)
14820
14821
14822def fluoview_description_metadata(description, ignoresections=None):
14823    r"""Return metatata from FluoView image description as dict.
14824
14825    The FluoView image description format is unspecified. Expect failures.
14826
14827    >>> descr = ('[Intensity Mapping]\nMap Ch0: Range=00000 to 02047\n'
14828    ...          '[Intensity Mapping End]')
14829    >>> fluoview_description_metadata(descr)
14830    {'Intensity Mapping': {'Map Ch0: Range': '00000 to 02047'}}
14831
14832    """
14833    if not description.startswith('['):
14834        raise ValueError('invalid FluoView image description')
14835    if ignoresections is None:
14836        ignoresections = {'Region Info (Fields)', 'Protocol Description'}
14837
14838    result = {}
14839    sections = [result]
14840    comment = False
14841    for line in description.splitlines():
14842        if not comment:
14843            line = line.strip()
14844        if not line:
14845            continue
14846        if line[0] == '[':
14847            if line[-5:] == ' End]':
14848                # close section
14849                del sections[-1]
14850                section = sections[-1]
14851                name = line[1:-5]
14852                if comment:
14853                    section[name] = '\n'.join(section[name])
14854                if name[:4] == 'LUT ':
14855                    a = numpy.array(section[name], dtype=numpy.uint8)
14856                    a.shape = -1, 3
14857                    section[name] = a
14858                continue
14859            # new section
14860            comment = False
14861            name = line[1:-1]
14862            if name[:4] == 'LUT ':
14863                section = []
14864            elif name in ignoresections:
14865                section = []
14866                comment = True
14867            else:
14868                section = {}
14869            sections.append(section)
14870            result[name] = section
14871            continue
14872        # add entry
14873        if comment:
14874            section.append(line)
14875            continue
14876        line = line.split('=', 1)
14877        if len(line) == 1:
14878            section[line[0].strip()] = None
14879            continue
14880        key, value = line
14881        if key[:4] == 'RGB ':
14882            section.extend(int(rgb) for rgb in value.split())
14883        else:
14884            section[key.strip()] = astype(value.strip())
14885    return result
14886
14887
14888def pilatus_description_metadata(description):
14889    """Return metatata from Pilatus image description as dict.
14890
14891    Return metadata from Pilatus pixel array detectors by Dectris, created
14892    by camserver or TVX software.
14893
14894    >>> pilatus_description_metadata('# Pixel_size 172e-6 m x 172e-6 m')
14895    {'Pixel_size': (0.000172, 0.000172)}
14896
14897    """
14898    result = {}
14899    if not description.startswith('# '):
14900        return result
14901    for c in '#:=,()':
14902        description = description.replace(c, ' ')
14903    for line in description.split('\n'):
14904        if line[:2] != '  ':
14905            continue
14906        line = line.split()
14907        name = line[0]
14908        if line[0] not in TIFF.PILATUS_HEADER:
14909            try:
14910                result['DateTime'] = datetime.datetime.strptime(
14911                    ' '.join(line), '%Y-%m-%dT%H %M %S.%f'
14912                )
14913            except Exception:
14914                result[name] = ' '.join(line[1:])
14915            continue
14916        indices, dtype = TIFF.PILATUS_HEADER[line[0]]
14917        if isinstance(indices[0], slice):
14918            # assumes one slice
14919            values = line[indices[0]]
14920        else:
14921            values = [line[i] for i in indices]
14922        if dtype is float and values[0] == 'not':
14923            values = ['NaN']
14924        values = tuple(dtype(v) for v in values)
14925        if dtype == str:
14926            values = ' '.join(values)
14927        elif len(values) == 1:
14928            values = values[0]
14929        result[name] = values
14930    return result
14931
14932
14933def svs_description_metadata(description):
14934    """Return metatata from Aperio image description as dict.
14935
14936    The Aperio image description format is unspecified. Expect failures.
14937
14938    >>> svs_description_metadata('Aperio Image Library v1.0')
14939    {'Header': 'Aperio Image Library v1.0'}
14940
14941    """
14942    if not description.startswith('Aperio '):
14943        raise ValueError('invalid Aperio image description')
14944    result = {}
14945    items = description.split('|')
14946    result['Header'] = items[0]
14947    if len(items) == 1:
14948        return result
14949    for item in items[1:]:
14950        key, value = item.split(' = ')
14951        result[key.strip()] = astype(value.strip())
14952    return result
14953
14954
14955def stk_description_metadata(description):
14956    """Return metadata from MetaMorph image description as list of dict.
14957
14958    The MetaMorph image description format is unspecified. Expect failures.
14959
14960    """
14961    description = description.strip()
14962    if not description:
14963        return []
14964    try:
14965        description = bytes2str(description)
14966    except UnicodeDecodeError as exc:
14967        log_warning(
14968            f'stk_description_metadata: {exc.__class__.__name__}: {exc}'
14969        )
14970        return []
14971    result = []
14972    for plane in description.split('\x00'):
14973        d = {}
14974        for line in plane.split('\r\n'):
14975            line = line.split(':', 1)
14976            if len(line) > 1:
14977                name, value = line
14978                d[name.strip()] = astype(value.strip())
14979            else:
14980                value = line[0].strip()
14981                if value:
14982                    if '' in d:
14983                        d[''].append(value)
14984                    else:
14985                        d[''] = [value]
14986        result.append(d)
14987    return result
14988
14989
14990def metaseries_description_metadata(description):
14991    """Return metatata from MetaSeries image description as dict."""
14992    if not description.startswith('<MetaData>'):
14993        raise ValueError('invalid MetaSeries image description')
14994
14995    from xml.etree import ElementTree as etree  # delayed import
14996
14997    root = etree.fromstring(description)
14998    types = {
14999        'float': float,
15000        'int': int,
15001        'bool': lambda x: asbool(x, 'on', 'off'),
15002    }
15003
15004    def parse(root, result):
15005        # recursive
15006        for child in root:
15007            attrib = child.attrib
15008            if not attrib:
15009                result[child.tag] = parse(child, {})
15010                continue
15011            if 'id' in attrib:
15012                i = attrib['id']
15013                t = attrib['type']
15014                v = attrib['value']
15015                if t in types:
15016                    result[i] = types[t](v)
15017                else:
15018                    result[i] = v
15019        return result
15020
15021    adict = parse(root, {})
15022    if 'Description' in adict:
15023        adict['Description'] = adict['Description'].replace('&#13;&#10;', '\n')
15024    return adict
15025
15026
15027def scanimage_description_metadata(description):
15028    """Return metatata from ScanImage image description as dict."""
15029    return matlabstr2py(description)
15030
15031
15032def scanimage_artist_metadata(artist):
15033    """Return metatata from ScanImage artist tag as dict."""
15034    try:
15035        return json.loads(artist)
15036    except ValueError as exc:
15037        log_warning(
15038            f'scanimage_artist_metadata: {exc.__class__.__name__}: {exc}'
15039        )
15040    return None
15041
15042
15043def olympusini_metadata(inistr):
15044    """Return OlympusSIS metadata from INI string.
15045
15046    No documentation is available.
15047
15048    """
15049
15050    def keyindex(key):
15051        # split key into name and index
15052        index = 0
15053        i = len(key.rstrip('0123456789'))
15054        if i < len(key):
15055            index = int(key[i:]) - 1
15056            key = key[:i]
15057        return key, index
15058
15059    result = {}
15060    bands = []
15061    zpos = None
15062    tpos = None
15063    for line in inistr.splitlines():
15064        line = line.strip()
15065        if line == '' or line[0] == ';':
15066            continue
15067        if line[0] == '[' and line[-1] == ']':
15068            section_name = line[1:-1]
15069            result[section_name] = section = {}
15070            if section_name == 'Dimension':
15071                result['axes'] = axes = []
15072                result['shape'] = shape = []
15073            elif section_name == 'ASD':
15074                result[section_name] = []
15075            elif section_name == 'Z':
15076                if 'Dimension' in result:
15077                    result[section_name]['ZPos'] = zpos = []
15078            elif section_name == 'Time':
15079                if 'Dimension' in result:
15080                    result[section_name]['TimePos'] = tpos = []
15081            elif section_name == 'Band':
15082                nbands = result['Dimension']['Band']
15083                bands = [{'LUT': []} for _ in range(nbands)]
15084                result[section_name] = bands
15085                iband = 0
15086        else:
15087            key, value = line.split('=')
15088            if value.strip() == '':
15089                value = None
15090            elif ',' in value:
15091                value = tuple(astype(v) for v in value.split(','))
15092            else:
15093                value = astype(value)
15094
15095            if section_name == 'Dimension':
15096                section[key] = value
15097                axes.append(key)
15098                shape.append(value)
15099            elif section_name == 'ASD':
15100                if key == 'Count':
15101                    result['ASD'] = [{}] * value
15102                else:
15103                    key, index = keyindex(key)
15104                    result['ASD'][index][key] = value
15105            elif section_name == 'Band':
15106                if key[:3] == 'LUT':
15107                    lut = bands[iband]['LUT']
15108                    value = struct.pack('<I', value)
15109                    lut.append(
15110                        [ord(value[0:1]), ord(value[1:2]), ord(value[2:3])]
15111                    )
15112                else:
15113                    key, iband = keyindex(key)
15114                    bands[iband][key] = value
15115            elif key[:4] == 'ZPos' and zpos is not None:
15116                zpos.append(value)
15117            elif key[:7] == 'TimePos' and tpos is not None:
15118                tpos.append(value)
15119            else:
15120                section[key] = value
15121
15122    if 'axes' in result:
15123        sisaxes = {'Band': 'C'}
15124        axes = []
15125        shape = []
15126        for i, x in zip(result['shape'], result['axes']):
15127            if i > 1:
15128                axes.append(sisaxes.get(x, x[0].upper()))
15129                shape.append(i)
15130        result['axes'] = ''.join(axes)
15131        result['shape'] = tuple(shape)
15132    try:
15133        result['Z']['ZPos'] = numpy.array(
15134            result['Z']['ZPos'][: result['Dimension']['Z']], 'float64'
15135        )
15136    except Exception:
15137        pass
15138    try:
15139        result['Time']['TimePos'] = numpy.array(
15140            result['Time']['TimePos'][: result['Dimension']['Time']], 'int32'
15141        )
15142    except Exception:
15143        pass
15144    for band in bands:
15145        band['LUT'] = numpy.array(band['LUT'], 'uint8')
15146    return result
15147
15148
15149def unpack_rgb(data, dtype=None, bitspersample=None, rescale=True):
15150    """Return array from bytes containing packed samples.
15151
15152    Use to unpack RGB565 or RGB555 to RGB888 format.
15153    Works on little-endian platforms only.
15154
15155    Parameters
15156    ----------
15157    data : byte str
15158        The data to be decoded. Samples in each pixel are stored consecutively.
15159        Pixels are aligned to 8, 16, or 32 bit boundaries.
15160    dtype : numpy.dtype
15161        The sample data type. The byteorder applies also to the data stream.
15162    bitspersample : tuple
15163        Number of bits for each sample in a pixel.
15164    rescale : bool
15165        Upscale samples to the number of bits in dtype.
15166
15167    Returns
15168    -------
15169    numpy.ndarray
15170        Flattened array of unpacked samples of native dtype.
15171
15172    Examples
15173    --------
15174    >>> data = struct.pack('BBBB', 0x21, 0x08, 0xff, 0xff)
15175    >>> print(unpack_rgb(data, '<B', (5, 6, 5), False))
15176    [ 1  1  1 31 63 31]
15177    >>> print(unpack_rgb(data, '<B', (5, 6, 5)))
15178    [  8   4   8 255 255 255]
15179    >>> print(unpack_rgb(data, '<B', (5, 5, 5)))
15180    [ 16   8   8 255 255 255]
15181
15182    """
15183    if bitspersample is None:
15184        bitspersample = (5, 6, 5)
15185    if dtype is None:
15186        dtype = '<B'
15187    dtype = numpy.dtype(dtype)
15188    bits = int(numpy.sum(bitspersample))
15189    if not (
15190        bits <= 32 and all(i <= dtype.itemsize * 8 for i in bitspersample)
15191    ):
15192        raise ValueError(f'sample size not supported: {bitspersample}')
15193    dt = next(i for i in 'BHI' if numpy.dtype(i).itemsize * 8 >= bits)
15194    data = numpy.frombuffer(data, dtype.byteorder + dt)
15195    result = numpy.empty((data.size, len(bitspersample)), dtype.char)
15196    for i, bps in enumerate(bitspersample):
15197        t = data >> int(numpy.sum(bitspersample[i + 1 :]))
15198        t &= int('0b' + '1' * bps, 2)
15199        if rescale:
15200            o = ((dtype.itemsize * 8) // bps + 1) * bps
15201            if o > data.dtype.itemsize * 8:
15202                t = t.astype('I')
15203            t *= (2 ** o - 1) // (2 ** bps - 1)
15204            t //= 2 ** (o - (dtype.itemsize * 8))
15205        result[:, i] = t
15206    return result.reshape(-1)
15207
15208
15209def float24_decode(data, byteorder):
15210    """Return float32 array from float24."""
15211    raise NotImplementedError('float24_decode')
15212
15213
15214def zlib_encode(data, level=None, out=None):
15215    """Compress Zlib DEFLATE."""
15216    import zlib
15217
15218    return zlib.compress(data, 6 if level is None else level)
15219
15220
15221def zlib_decode(data, out=None):
15222    """Decompress Zlib DEFLATE."""
15223    import zlib
15224
15225    return zlib.decompress(data)
15226
15227
15228def lzma_encode(data, level=None, out=None):
15229    """Compress LZMA."""
15230    import lzma
15231
15232    return lzma.compress(data)
15233
15234
15235def lzma_decode(data, out=None):
15236    """Decompress LZMA."""
15237    import lzma
15238
15239    return lzma.decompress(data)
15240
15241
15242if imagecodecs is None:
15243
15244    def delta_encode(data, axis=-1, dist=1, out=None):
15245        """Encode Delta."""
15246        if dist != 1:
15247            raise NotImplementedError(f'dist {dist} not implemented')
15248        if isinstance(data, (bytes, bytearray)):
15249            data = numpy.frombuffer(data, dtype=numpy.uint8)
15250            diff = numpy.diff(data, axis=0)
15251            return numpy.insert(diff, 0, data[0]).tobytes()
15252
15253        dtype = data.dtype
15254        if dtype.kind == 'f':
15255            data = data.view(f'u{dtype.itemsize}')
15256
15257        diff = numpy.diff(data, axis=axis)
15258        key = [slice(None)] * data.ndim
15259        key[axis] = 0
15260        diff = numpy.insert(diff, 0, data[tuple(key)], axis=axis)
15261
15262        if dtype.kind == 'f':
15263            return diff.view(dtype)
15264        return diff
15265
15266    def delta_decode(data, axis=-1, dist=1, out=None):
15267        """Decode Delta."""
15268        if dist != 1:
15269            raise NotImplementedError(f'dist {dist} not implemented')
15270        if out is not None and not out.flags.writeable:
15271            out = None
15272        if isinstance(data, (bytes, bytearray)):
15273            data = numpy.frombuffer(data, dtype=numpy.uint8)
15274            return numpy.cumsum(
15275                data, axis=0, dtype=numpy.uint8, out=out
15276            ).tobytes()
15277        if data.dtype.kind == 'f':
15278            view = data.view(f'u{data.dtype.itemsize}')
15279            view = numpy.cumsum(view, axis=axis, dtype=view.dtype)
15280            return view.view(data.dtype)
15281        return numpy.cumsum(data, axis=axis, dtype=data.dtype, out=out)
15282
15283    def bitorder_decode(data, out=None, _bitorder=[]):
15284        r"""Reverse bits in each byte of bytes or numpy array.
15285
15286        Decode data where pixels with lower column values are stored in the
15287        lower-order bits of the bytes (TIFF FillOrder is LSB2MSB).
15288
15289        Parameters
15290        ----------
15291        data : bytes or ndarray
15292            The data to be bit reversed. If bytes, a new bit-reversed
15293            bytes is returned. Numpy arrays are bit-reversed in-place.
15294
15295        Examples
15296        --------
15297        >>> bitorder_decode(b'\x01\x64')
15298        b'\x80&'
15299        >>> data = numpy.array([1, 666], dtype='uint16')
15300        >>> bitorder_decode(data)
15301        >>> data
15302        array([  128, 16473], dtype=uint16)
15303
15304        """
15305        if not _bitorder:
15306            _bitorder.append(
15307                b'\x00\x80@\xc0 \xa0`\xe0\x10\x90P\xd00\xb0p\xf0\x08\x88H'
15308                b'\xc8(\xa8h\xe8\x18\x98X\xd88\xb8x\xf8\x04\x84D\xc4$\xa4d'
15309                b'\xe4\x14\x94T\xd44\xb4t\xf4\x0c\x8cL\xcc,\xacl\xec\x1c\x9c'
15310                b'\\\xdc<\xbc|\xfc\x02\x82B\xc2"\xa2b\xe2\x12\x92R\xd22'
15311                b'\xb2r\xf2\n\x8aJ\xca*\xaaj\xea\x1a\x9aZ\xda:\xbaz\xfa'
15312                b'\x06\x86F\xc6&\xa6f\xe6\x16\x96V\xd66\xb6v\xf6\x0e\x8eN'
15313                b'\xce.\xaen\xee\x1e\x9e^\xde>\xbe~\xfe\x01\x81A\xc1!\xa1a'
15314                b'\xe1\x11\x91Q\xd11\xb1q\xf1\t\x89I\xc9)\xa9i\xe9\x19'
15315                b'\x99Y\xd99\xb9y\xf9\x05\x85E\xc5%\xa5e\xe5\x15\x95U\xd55'
15316                b'\xb5u\xf5\r\x8dM\xcd-\xadm\xed\x1d\x9d]\xdd=\xbd}\xfd'
15317                b'\x03\x83C\xc3#\xa3c\xe3\x13\x93S\xd33\xb3s\xf3\x0b\x8bK'
15318                b'\xcb+\xabk\xeb\x1b\x9b[\xdb;\xbb{\xfb\x07\x87G\xc7\'\xa7g'
15319                b'\xe7\x17\x97W\xd77\xb7w\xf7\x0f\x8fO\xcf/\xafo\xef\x1f\x9f_'
15320                b'\xdf?\xbf\x7f\xff'
15321            )
15322            _bitorder.append(numpy.frombuffer(_bitorder[0], dtype=numpy.uint8))
15323        try:
15324            view = data.view('uint8')
15325            numpy.take(_bitorder[1], view, out=view)
15326            return data
15327        except AttributeError:
15328            return data.translate(_bitorder[0])
15329        except ValueError:
15330            raise NotImplementedError('slices of arrays not supported')
15331        return None
15332
15333    def packints_encode(data, bitspersample, axis=-1, out=None):
15334        """Tightly pack integers."""
15335        raise NotImplementedError('packints_encode')
15336
15337    def packints_decode(data, dtype, bitspersample, runlen=0, out=None):
15338        """Decompress bytes to array of integers.
15339
15340        This implementation only handles itemsizes 1, 8, 16, 32, and 64 bits.
15341        Install the imagecodecs package for decoding other integer sizes.
15342
15343        Parameters
15344        ----------
15345        data : byte str
15346            Data to decompress.
15347        dtype : numpy.dtype or str
15348            A numpy boolean or integer type.
15349        bitspersample : int
15350            Number of bits per integer.
15351        runlen : int
15352            Number of consecutive integers, after which to start at next byte.
15353
15354        Examples
15355        --------
15356        >>> packints_decode(b'a', 'B', 1)
15357        array([0, 1, 1, 0, 0, 0, 0, 1], dtype=uint8)
15358
15359        """
15360        if bitspersample == 1:  # bitarray
15361            data = numpy.frombuffer(data, '|B')
15362            data = numpy.unpackbits(data)
15363            if runlen % 8:
15364                data = data.reshape(-1, runlen + (8 - runlen % 8))
15365                data = data[:, :runlen].reshape(-1)
15366            return data.astype(dtype)
15367        if bitspersample in (8, 16, 32, 64):
15368            return numpy.frombuffer(data, dtype)
15369        raise NotImplementedError(
15370            f'unpacking {bitspersample}-bit integers '
15371            f'to {numpy.dtype(dtype)} not supported'
15372        )
15373
15374    def packbits_decode(encoded, out=None):
15375        r"""Decompress PackBits encoded byte string.
15376
15377        >>> packbits_decode(b'\x80\x80')  # NOP
15378        b''
15379        >>> packbits_decode(b'\x02123')
15380        b'123'
15381        >>> packbits_decode(
15382        ...   b'\xfe\xaa\x02\x80\x00\x2a\xfd\xaa\x03\x80\x00\x2a\x22\xf7\xaa'
15383        ...     )[:-5]
15384        b'\xaa\xaa\xaa\x80\x00*\xaa\xaa\xaa\xaa\x80\x00*"\xaa\xaa\xaa\xaa\xaa'
15385
15386        """
15387        out = []
15388        out_extend = out.extend
15389        i = 0
15390        try:
15391            while True:
15392                n = ord(encoded[i : i + 1]) + 1
15393                i += 1
15394                if n > 129:
15395                    # replicate
15396                    out_extend(encoded[i : i + 1] * (258 - n))
15397                    i += 1
15398                elif n < 129:
15399                    # literal
15400                    out_extend(encoded[i : i + n])
15401                    i += n
15402        except TypeError:
15403            pass
15404        return bytes(out)
15405
15406
15407else:
15408    bitorder_decode = imagecodecs.bitorder_decode  # noqa
15409    packints_decode = imagecodecs.packints_decode  # noqa
15410    packints_encode = imagecodecs.packints_encode  # noqa
15411    try:
15412        float24_decode = imagecodecs.float24_decode  # noqa
15413    except AttributeError:
15414        pass
15415
15416
15417def apply_colormap(image, colormap, contig=True):
15418    """Return palette-colored image.
15419
15420    The image values are used to index the colormap on axis 1. The returned
15421    image is of shape image.shape+colormap.shape[0] and dtype colormap.dtype.
15422
15423    Parameters
15424    ----------
15425    image : numpy.ndarray
15426        Indexes into the colormap.
15427    colormap : numpy.ndarray
15428        RGB lookup table aka palette of shape (3, 2**bits_per_sample).
15429    contig : bool
15430        If True, return a contiguous array.
15431
15432    Examples
15433    --------
15434    >>> image = numpy.arange(256, dtype='uint8')
15435    >>> colormap = numpy.vstack([image, image, image]).astype('uint16') * 256
15436    >>> apply_colormap(image, colormap)[-1]
15437    array([65280, 65280, 65280], dtype=uint16)
15438
15439    """
15440    image = numpy.take(colormap, image, axis=1)
15441    image = numpy.rollaxis(image, 0, image.ndim)
15442    if contig:
15443        image = numpy.ascontiguousarray(image)
15444    return image
15445
15446
15447def parse_filenames(files, pattern, axesorder=None):
15448    r"""Return shape and axes from sequence of file names matching pattern.
15449
15450    >>> parse_filenames(
15451    ...     ['c1001.ext', 'c2002.ext'], r'([^\d])(\d)(?P<t>\d+)\.ext'
15452    ... )
15453    ('ct', (2, 2), [(0, 0), (1, 1)], (1, 1))
15454
15455    """
15456    if not pattern:
15457        raise ValueError('invalid pattern')
15458    if isinstance(pattern, str):
15459        pattern = re.compile(pattern)
15460
15461    def parse(fname, pattern=pattern):
15462        # return axes and indices from file name
15463        # fname = os.path.split(fname)[-1]
15464        axes = []
15465        indices = []
15466        groupindex = {v: k for k, v in pattern.groupindex.items()}
15467        match = pattern.search(fname)
15468        if not match:
15469            raise ValueError('pattern does not match file name')
15470        ax = None
15471        for i, m in enumerate(match.groups()):
15472            if m is None:
15473                continue
15474            if i + 1 in groupindex:
15475                ax = groupindex[i + 1]  # names axis
15476                if not m[0].isdigit():
15477                    m = ord(m)  # index letter to number
15478                    if m < 65 or m > 122:
15479                        raise ValueError(f'invalid index {m!r}')
15480            elif m[0].isalpha():
15481                ax = m  # axis letter for next index
15482                continue
15483            if ax is None:
15484                ax = 'Q'  # no preceding axis letter
15485            try:
15486                m = int(m)
15487            except Exception:
15488                raise ValueError(f'invalid index {m!r}')
15489            indices.append(m)
15490            axes.append(ax)
15491            ax = None
15492        return ''.join(axes), indices
15493
15494    files = [os.path.normpath(f) for f in files]
15495    if len(files) == 1:
15496        prefix = os.path.dirname(files[0])
15497    else:
15498        prefix = os.path.commonpath(files)
15499    prefix = len(prefix)
15500
15501    axes = None
15502    indices = []
15503    for fname in files:
15504        ax, idx = parse(fname[prefix:])
15505        if axes is None:
15506            axes = ax
15507            if axesorder is not None and (
15508                len(axesorder) != len(axes)
15509                or any(i not in axesorder for i in range(len(axes)))
15510            ):
15511                raise ValueError('invalid axisorder')
15512        elif axes != ax:
15513            raise ValueError('axes do not match within image sequence')
15514        if axesorder is not None:
15515            idx = [idx[i] for i in axesorder]
15516        indices.append(idx)
15517
15518    if axesorder is not None:
15519        axes = ''.join(axes[i] for i in axesorder)
15520
15521    indices = numpy.array(indices, dtype=numpy.intp)
15522    startindex = numpy.min(indices, axis=0)
15523    shape = numpy.max(indices, axis=0)
15524    shape -= startindex
15525    shape += 1
15526    shape = tuple(shape.tolist())
15527    indices -= startindex
15528    indices = indices.tolist()
15529    indices = [tuple(index) for index in indices]
15530    startindex = tuple(startindex.tolist())
15531    return axes, shape, indices, startindex
15532
15533
15534def iter_images(data):
15535    """Return iterator over pages in data array of normalized shape."""
15536    yield from data
15537
15538
15539def iter_tiles(data, tile, tiles):
15540    """Return iterator over tiles in data array of normalized shape."""
15541    shape = data.shape
15542    chunk = numpy.empty(tile + (shape[-1],), dtype=data.dtype)
15543    if not 1 < len(tile) < 4:
15544        raise ValueError('invalid tile shape')
15545    if len(tile) == 2:
15546        for page in data:
15547            for plane in page:
15548                for ty in range(tiles[0]):
15549                    for tx in range(tiles[1]):
15550                        c1 = min(tile[0], shape[3] - ty * tile[0])
15551                        c2 = min(tile[1], shape[4] - tx * tile[1])
15552                        chunk[c1:, c2:] = 0
15553                        chunk[:c1, :c2] = plane[
15554                            0,
15555                            ty * tile[0] : ty * tile[0] + c1,
15556                            tx * tile[1] : tx * tile[1] + c2,
15557                        ]
15558                        yield chunk
15559    else:
15560        for page in data:
15561            for plane in page:
15562                for tz in range(tiles[0]):
15563                    for ty in range(tiles[1]):
15564                        for tx in range(tiles[2]):
15565                            c0 = min(tile[0], shape[2] - tz * tile[0])
15566                            c1 = min(tile[1], shape[3] - ty * tile[1])
15567                            c2 = min(tile[2], shape[4] - tx * tile[2])
15568                            chunk[c0:, c1:, c2:] = 0
15569                            chunk[:c0, :c1, :c2] = plane[
15570                                tz * tile[0] : tz * tile[0] + c0,
15571                                ty * tile[1] : ty * tile[1] + c1,
15572                                tx * tile[2] : tx * tile[2] + c2,
15573                            ]
15574                            if tile[0] == 1:
15575                                # squeeze for image compressors
15576                                yield chunk[0]
15577                            else:
15578                                yield chunk
15579
15580
15581def pad_tile(tile, shape, dtype):
15582    """Return tile padded to tile shape."""
15583    if tile.dtype != dtype or tile.nbytes > product(shape) * dtype.itemsize:
15584        raise ValueError('invalid tile shape or dtype')
15585    pad = tuple((0, i - j) for i, j in zip(shape, tile.shape))
15586    return numpy.pad(tile, pad)
15587
15588
15589def reorient(image, orientation):
15590    """Return reoriented view of image array.
15591
15592    Parameters
15593    ----------
15594    image : numpy.ndarray
15595        Non-squeezed output of asarray() functions.
15596        Axes -3 and -2 must be image length and width respectively.
15597    orientation : int or str
15598        One of TIFF.ORIENTATION names or values.
15599
15600    """
15601    orient = TIFF.ORIENTATION
15602    orientation = enumarg(orient, orientation)
15603
15604    if orientation == orient.TOPLEFT:
15605        return image
15606    if orientation == orient.TOPRIGHT:
15607        return image[..., ::-1, :]
15608    if orientation == orient.BOTLEFT:
15609        return image[..., ::-1, :, :]
15610    if orientation == orient.BOTRIGHT:
15611        return image[..., ::-1, ::-1, :]
15612    if orientation == orient.LEFTTOP:
15613        return numpy.swapaxes(image, -3, -2)
15614    if orientation == orient.RIGHTTOP:
15615        return numpy.swapaxes(image, -3, -2)[..., ::-1, :]
15616    if orientation == orient.RIGHTBOT:
15617        return numpy.swapaxes(image, -3, -2)[..., ::-1, :, :]
15618    if orientation == orient.LEFTBOT:
15619        return numpy.swapaxes(image, -3, -2)[..., ::-1, ::-1, :]
15620    return image
15621
15622
15623def repeat_nd(a, repeats):
15624    """Return read-only view into input array with elements repeated.
15625
15626    Zoom nD image by integer factors using nearest neighbor interpolation
15627    (box filter).
15628
15629    Parameters
15630    ----------
15631    a : array-like
15632        Input array.
15633    repeats : sequence of int
15634        The number of repetitions to apply along each dimension of input array.
15635
15636    Examples
15637    --------
15638    >>> repeat_nd([[1, 2], [3, 4]], (2, 2))
15639    array([[1, 1, 2, 2],
15640           [1, 1, 2, 2],
15641           [3, 3, 4, 4],
15642           [3, 3, 4, 4]])
15643
15644    """
15645    a = numpy.asarray(a)
15646    reshape = []
15647    shape = []
15648    strides = []
15649    for i, j, k in zip(a.strides, a.shape, repeats):
15650        shape.extend((j, k))
15651        strides.extend((i, 0))
15652        reshape.append(j * k)
15653    return numpy.lib.stride_tricks.as_strided(
15654        a, shape, strides, writeable=False
15655    ).reshape(reshape)
15656
15657
15658def reshape_nd(data_or_shape, ndim):
15659    """Return image array or shape with at least ndim dimensions.
15660
15661    Prepend 1s to image shape as necessary.
15662
15663    >>> reshape_nd(numpy.empty(0), 1).shape
15664    (0,)
15665    >>> reshape_nd(numpy.empty(1), 2).shape
15666    (1, 1)
15667    >>> reshape_nd(numpy.empty((2, 3)), 3).shape
15668    (1, 2, 3)
15669    >>> reshape_nd(numpy.empty((3, 4, 5)), 3).shape
15670    (3, 4, 5)
15671    >>> reshape_nd((2, 3), 3)
15672    (1, 2, 3)
15673
15674    """
15675    is_shape = isinstance(data_or_shape, tuple)
15676    shape = data_or_shape if is_shape else data_or_shape.shape
15677    if len(shape) >= ndim:
15678        return data_or_shape
15679    shape = (1,) * (ndim - len(shape)) + shape
15680    return shape if is_shape else data_or_shape.reshape(shape)
15681
15682
15683def squeeze_axes(shape, axes, skip=None):
15684    """Return shape and axes with single-dimensional entries removed.
15685
15686    Remove unused dimensions unless their axes are listed in 'skip'.
15687
15688    >>> squeeze_axes((5, 1, 2, 1, 1), 'TZYXC')
15689    ((5, 2, 1), 'TYX')
15690
15691    >>> squeeze_axes((1,), 'Q')
15692    ((1,), 'Q')
15693
15694    """
15695    if len(shape) != len(axes):
15696        raise ValueError('dimensions of axes and shape do not match')
15697    if skip is None:
15698        skip = 'XY'
15699    try:
15700        shape_squeezed, axes_squeezed = zip(
15701            *(i for i in zip(shape, axes) if i[0] > 1 or i[1] in skip)
15702        )
15703    except ValueError:
15704        # not enough values to unpack, return last axis
15705        shape_squeezed = shape[-1:]
15706        axes_squeezed = axes[-1:]
15707    return tuple(shape_squeezed), ''.join(axes_squeezed)
15708
15709
15710def transpose_axes(image, axes, asaxes=None):
15711    """Return image with its axes permuted to match specified axes.
15712
15713    A view is returned if possible.
15714
15715    >>> transpose_axes(numpy.zeros((2, 3, 4, 5)), 'TYXC', asaxes='CTZYX').shape
15716    (5, 2, 1, 3, 4)
15717
15718    """
15719    for ax in axes:
15720        if ax not in asaxes:
15721            raise ValueError(f'unknown axis {ax}')
15722    # add missing axes to image
15723    if asaxes is None:
15724        asaxes = 'CTZYX'
15725    shape = image.shape
15726    for ax in reversed(asaxes):
15727        if ax not in axes:
15728            axes = ax + axes
15729            shape = (1,) + shape
15730    image = image.reshape(shape)
15731    # transpose axes
15732    image = image.transpose([axes.index(ax) for ax in asaxes])
15733    return image
15734
15735
15736def reshape_axes(axes, shape, newshape, unknown=None):
15737    """Return axes matching new shape.
15738
15739    By default, unknown dimensions are labelled 'Q'.
15740
15741    >>> reshape_axes('YXS', (219, 301, 1), (219, 301))
15742    'YX'
15743    >>> reshape_axes('IYX', (12, 219, 301), (3, 4, 219, 1, 301, 1))
15744    'QQYQXQ'
15745
15746    """
15747    shape = tuple(shape)
15748    newshape = tuple(newshape)
15749    if len(axes) != len(shape):
15750        raise ValueError('axes do not match shape')
15751
15752    size = product(shape)
15753    newsize = product(newshape)
15754    if size != newsize:
15755        raise ValueError(f'cannot reshape {shape} to {newshape}')
15756    if not axes or not newshape:
15757        return ''
15758
15759    lendiff = max(0, len(shape) - len(newshape))
15760    if lendiff:
15761        newshape = newshape + (1,) * lendiff
15762
15763    i = len(shape) - 1
15764    prodns = 1
15765    prods = 1
15766    result = []
15767    for ns in newshape[::-1]:
15768        prodns *= ns
15769        while i > 0 and shape[i] == 1 and ns != 1:
15770            i -= 1
15771        if ns == shape[i] and prodns == prods * shape[i]:
15772            prods *= shape[i]
15773            result.append(axes[i])
15774            i -= 1
15775        elif unknown:
15776            result.append(unknown)
15777        else:
15778            unknown = 'Q'
15779            result.append(unknown)
15780
15781    return ''.join(reversed(result[lendiff:]))
15782
15783
15784def subresolution(a, b, p=2, n=16):
15785    """Return level of subresolution of series or page b vs a."""
15786    if a.axes != b.axes or a.dtype != b.dtype:
15787        return None
15788    level = None
15789    for ax, i, j in zip(a.axes.lower(), a.shape, b.shape):
15790        if ax in 'xyz':
15791            if level is None:
15792                for r in range(n):
15793                    d = p ** r
15794                    if d > i:
15795                        return None
15796                    if abs((i / d) - j) < 1.0:
15797                        level = r
15798                        break
15799                else:
15800                    return None
15801            else:
15802                d = p ** level
15803                if d > i:
15804                    return None
15805                if abs((i / d) - j) >= 1.0:
15806                    return None
15807        elif i != j:
15808            return None
15809    return level
15810
15811
15812def pyramidize_series(series, isreduced=False):
15813    """Pyramidize list of TiffPageSeries in-place.
15814
15815    TiffPageSeries that are a subresolution of another TiffPageSeries are
15816    appended to the other's TiffPageSeries levels and removed from the list.
15817    Levels are to be ordered by size using the same downsampling factor.
15818    TiffPageSeries of subifds cannot be pyramid top levels.
15819
15820    """
15821    samplingfactors = (2, 3, 4)
15822    i = 0
15823    while i < len(series):
15824        a = series[i]
15825        p = None
15826        j = i + 1
15827        if isinstance(a.keyframe.index, tuple):
15828            # subifds cannot be pyramid top levels
15829            i += 1
15830            continue
15831        while j < len(series):
15832            b = series[j]
15833            if isreduced and not b.keyframe.is_reduced:
15834                # pyramid levels must be reduced
15835                j += 1
15836                continue  # not a pyramid level
15837            if p is None:
15838                for f in samplingfactors:
15839                    if subresolution(a.levels[-1], b, p=f) == 1:
15840                        p = f
15841                        break  # not a pyramid level
15842                else:
15843                    j += 1
15844                    continue  # not a pyramid level
15845            elif subresolution(a.levels[-1], b, p=p) != 1:
15846                j += 1
15847                continue
15848            a.levels.append(b)
15849            del series[j]
15850        i += 1
15851
15852
15853def stack_pages(pages, out=None, maxworkers=None, **kwargs):
15854    """Read data from sequence of TiffPage and stack them vertically.
15855
15856    Additional parameters are passsed to the TiffPage.asarray function.
15857
15858    """
15859    npages = len(pages)
15860    if npages == 0:
15861        raise ValueError('no pages')
15862
15863    if npages == 1:
15864        kwargs['maxworkers'] = maxworkers
15865        return pages[0].asarray(out=out, **kwargs)
15866
15867    page0 = next(p.keyframe for p in pages if p is not None)
15868    shape = (npages,) + page0.shape
15869    dtype = page0.dtype
15870    out = create_output(out, shape, dtype)
15871
15872    # TODO: benchmark and optimize this
15873    if maxworkers is None or maxworkers < 1:
15874        # auto-detect
15875        page_maxworkers = page0.maxworkers
15876        maxworkers = min(npages, TIFF.MAXWORKERS)
15877        if maxworkers == 1 or page_maxworkers < 1:
15878            maxworkers = page_maxworkers = 1
15879        elif npages < 3:
15880            maxworkers = 1
15881        elif (
15882            page_maxworkers <= 2
15883            and page0.compression == 1
15884            and page0.fillorder == 1
15885            and page0.predictor == 1
15886        ):
15887            maxworkers = 1
15888        else:
15889            page_maxworkers = 1
15890    elif maxworkers == 1:
15891        maxworkers = page_maxworkers = 1
15892    elif npages > maxworkers or page0.maxworkers < 2:
15893        page_maxworkers = 1
15894    else:
15895        page_maxworkers = maxworkers
15896        maxworkers = 1
15897
15898    kwargs['maxworkers'] = page_maxworkers
15899
15900    filehandle = page0.parent.filehandle
15901    haslock = filehandle.has_lock
15902    if not haslock and maxworkers > 1 or page_maxworkers > 1:
15903        filehandle.lock = True
15904    filecache = FileCache(size=max(4, maxworkers), lock=filehandle.lock)
15905
15906    def func(page, index, out=out, filecache=filecache, kwargs=kwargs):
15907        # read, decode, and copy page data
15908        if page is not None:
15909            filecache.open(page.parent.filehandle)
15910            page.asarray(lock=filecache.lock, out=out[index], **kwargs)
15911            filecache.close(page.parent.filehandle)
15912
15913    if maxworkers < 2:
15914        for i, page in enumerate(pages):
15915            func(page, i)
15916    else:
15917        page0.decode  # init TiffPage.decode function
15918        with ThreadPoolExecutor(maxworkers) as executor:
15919            for _ in executor.map(func, pages, range(npages)):
15920                pass
15921
15922    filecache.clear()
15923    if not haslock:
15924        filehandle.lock = False
15925    return out
15926
15927
15928def create_output(out, shape, dtype, mode='w+', suffix=None):
15929    """Return numpy array where image data of shape and dtype can be copied.
15930
15931    The 'out' parameter may have the following values or types:
15932
15933    None
15934        An empty array of shape and dtype is created and returned.
15935    numpy.ndarray
15936        An existing writable array of compatible dtype and shape. A view of
15937        the same array is returned after verification.
15938    'memmap' or 'memmap:tempdir'
15939        A memory-map to an array stored in a temporary binary file on disk
15940        is created and returned.
15941    str or open file
15942        The file name or file object used to create a memory-map to an array
15943        stored in a binary file on disk. The created memory-mapped array is
15944        returned.
15945
15946    """
15947    if out is None:
15948        return numpy.zeros(shape, dtype)
15949    if isinstance(out, numpy.ndarray):
15950        if product(shape) != product(out.shape):
15951            raise ValueError('incompatible output shape')
15952        if not numpy.can_cast(dtype, out.dtype):
15953            raise ValueError('incompatible output dtype')
15954        return out.reshape(shape)
15955    if isinstance(out, str) and out[:6] == 'memmap':
15956        import tempfile
15957
15958        tempdir = out[7:] if len(out) > 7 else None
15959        if suffix is None:
15960            suffix = '.memmap'
15961        with tempfile.NamedTemporaryFile(dir=tempdir, suffix=suffix) as fh:
15962            return numpy.memmap(fh, shape=shape, dtype=dtype, mode=mode)
15963    return numpy.memmap(out, shape=shape, dtype=dtype, mode=mode)
15964
15965
15966def matlabstr2py(string):
15967    r"""Return Python object from Matlab string representation.
15968
15969    Return str, bool, int, float, list (Matlab arrays or cells), or
15970    dict (Matlab structures) types.
15971
15972    Use to access ScanImage metadata.
15973
15974    >>> matlabstr2py('1')
15975    1
15976    >>> matlabstr2py("['x y z' true false; 1 2.0 -3e4; NaN Inf @class]")
15977    [['x y z', True, False], [1, 2.0, -30000.0], [nan, inf, '@class']]
15978    >>> d = matlabstr2py(
15979    ...     "SI.hChannels.channelType = {'stripe' 'stripe'}\n"
15980    ...     "SI.hChannels.channelsActive = 2"
15981    ... )
15982    >>> d['SI.hChannels.channelType']
15983    ['stripe', 'stripe']
15984
15985    """
15986    # TODO: handle invalid input
15987    # TODO: review unboxing of multidimensional arrays
15988
15989    def lex(s):
15990        # return sequence of tokens from matlab string representation
15991        tokens = ['[']
15992        while True:
15993            t, i = next_token(s)
15994            if t is None:
15995                break
15996            if t == ';':
15997                tokens.extend((']', '['))
15998            elif t == '[':
15999                tokens.extend(('[', '['))
16000            elif t == ']':
16001                tokens.extend((']', ']'))
16002            else:
16003                tokens.append(t)
16004            s = s[i:]
16005        tokens.append(']')
16006        return tokens
16007
16008    def next_token(s):
16009        # return next token in matlab string
16010        length = len(s)
16011        if length == 0:
16012            return None, 0
16013        i = 0
16014        while i < length and s[i] == ' ':
16015            i += 1
16016        if i == length:
16017            return None, i
16018        if s[i] in '{[;]}':
16019            return s[i], i + 1
16020        if s[i] == "'":
16021            j = i + 1
16022            while j < length and s[j] != "'":
16023                j += 1
16024            return s[i : j + 1], j + 1
16025        if s[i] == '<':
16026            j = i + 1
16027            while j < length and s[j] != '>':
16028                j += 1
16029            return s[i : j + 1], j + 1
16030        j = i
16031        while j < length and not s[j] in ' {[;]}':
16032            j += 1
16033        return s[i:j], j
16034
16035    def value(s, fail=False):
16036        # return Python value of token
16037        s = s.strip()
16038        if not s:
16039            return s
16040        if len(s) == 1:
16041            try:
16042                return int(s)
16043            except Exception:
16044                if fail:
16045                    raise ValueError()
16046                return s
16047        if s[0] == "'":
16048            if fail and s[-1] != "'" or "'" in s[1:-1]:
16049                raise ValueError()
16050            return s[1:-1]
16051        if s[0] == '<':
16052            if fail and s[-1] != '>' or '<' in s[1:-1]:
16053                raise ValueError()
16054            return s
16055        if fail and any(i in s for i in " ';[]{}"):
16056            raise ValueError()
16057        if s[0] == '@':
16058            return s
16059        if s in ('true', 'True'):
16060            return True
16061        if s in ('false', 'False'):
16062            return False
16063        if s[:6] == 'zeros(':
16064            return numpy.zeros([int(i) for i in s[6:-1].split(',')]).tolist()
16065        if s[:5] == 'ones(':
16066            return numpy.ones([int(i) for i in s[5:-1].split(',')]).tolist()
16067        if '.' in s or 'e' in s:
16068            try:
16069                return float(s)
16070            except Exception:
16071                pass
16072        try:
16073            return int(s)
16074        except Exception:
16075            pass
16076        try:
16077            return float(s)  # nan, inf
16078        except Exception:
16079            if fail:
16080                raise ValueError()
16081        return s
16082
16083    def parse(s):
16084        # return Python value from string representation of Matlab value
16085        s = s.strip()
16086        try:
16087            return value(s, fail=True)
16088        except ValueError:
16089            pass
16090        result = add2 = []
16091        levels = [add2]
16092        for t in lex(s):
16093            if t in '[{':
16094                add2 = []
16095                levels.append(add2)
16096            elif t in ']}':
16097                x = levels.pop()
16098                if len(x) == 1 and isinstance(x[0], (list, str)):
16099                    x = x[0]
16100                add2 = levels[-1]
16101                add2.append(x)
16102            else:
16103                add2.append(value(t))
16104        if len(result) == 1 and isinstance(result[0], (list, str)):
16105            result = result[0]
16106        return result
16107
16108    if '\r' in string or '\n' in string:
16109        # structure
16110        d = {}
16111        for line in string.splitlines():
16112            line = line.strip()
16113            if not line or line[0] == '%':
16114                continue
16115            k, v = line.split('=', 1)
16116            k = k.strip()
16117            if any(c in k for c in " ';[]{}<>"):
16118                continue
16119            d[k] = parse(v)
16120        return d
16121    return parse(string)
16122
16123
16124def stripnull(string, null=b'\x00', first=True):
16125    r"""Return string truncated at first null character.
16126
16127    Clean NULL terminated C strings. For Unicode strings use null='\0'.
16128
16129    >>> stripnull(b'string\x00\x00')
16130    b'string'
16131    >>> stripnull(b'string\x00string\x00\x00', first=False)
16132    b'string\x00string'
16133    >>> stripnull('string\x00', null='\0')
16134    'string'
16135
16136    """
16137    if first:
16138        i = string.find(null)
16139        return string if i < 0 else string[:i]
16140    null = null[0]
16141    i = len(string)
16142    while i:
16143        i -= 1
16144        if string[i] != null:
16145            break
16146    else:
16147        i = -1
16148    return string[: i + 1]
16149
16150
16151def stripascii(string):
16152    r"""Return string truncated at last byte that is 7-bit ASCII.
16153
16154    Clean NULL separated and terminated TIFF strings.
16155
16156    >>> stripascii(b'string\x00string\n\x01\x00')
16157    b'string\x00string\n'
16158    >>> stripascii(b'\x00')
16159    b''
16160
16161    """
16162    # TODO: pythonize this
16163    i = len(string)
16164    while i:
16165        i -= 1
16166        if 8 < string[i] < 127:
16167            break
16168    else:
16169        i = -1
16170    return string[: i + 1]
16171
16172
16173def asbool(value, true=None, false=None):
16174    """Return string as bool if possible, else raise TypeError.
16175
16176    >>> asbool(b' False ')
16177    False
16178    >>> asbool('ON', ['on'], ['off'])
16179    True
16180
16181    """
16182    value = value.strip().lower()
16183    isbytes = False
16184    if true is None:
16185        if isinstance(value, bytes):
16186            if value == b'true':
16187                return True
16188            isbytes = True
16189        elif value == 'true':
16190            return True
16191    if false is None:
16192        if isbytes or isinstance(value, bytes):
16193            if value == b'false':
16194                return False
16195        elif value == 'false':
16196            return False
16197    if value in true:
16198        return True
16199    if value in false:
16200        return False
16201    raise TypeError()
16202
16203
16204def astype(value, types=None):
16205    """Return argument as one of types if possible.
16206
16207    >>> astype('42')
16208    42
16209    >>> astype('3.14')
16210    3.14
16211    >>> astype('True')
16212    True
16213    >>> astype(b'Neee-Wom')
16214    'Neee-Wom'
16215
16216    """
16217    if types is None:
16218        types = int, float, asbool, bytes2str
16219    for typ in types:
16220        try:
16221            return typ(value)
16222        except (ValueError, AttributeError, TypeError, UnicodeEncodeError):
16223            pass
16224    return value
16225
16226
16227def format_size(size, threshold=1536):
16228    """Return file size as string from byte size.
16229
16230    >>> format_size(1234)
16231    '1234 B'
16232    >>> format_size(12345678901)
16233    '11.50 GiB'
16234
16235    """
16236    if size < threshold:
16237        return f'{size} B'
16238    for unit in ('KiB', 'MiB', 'GiB', 'TiB', 'PiB'):
16239        size /= 1024.0
16240        if size < threshold:
16241            return f'{size:.2f} {unit}'
16242    return 'ginormous'
16243
16244
16245def identityfunc(arg, *args, **kwargs):
16246    """Single argument identity function.
16247
16248    >>> identityfunc('arg')
16249    'arg'
16250
16251    """
16252    return arg
16253
16254
16255def nullfunc(*args, **kwargs):
16256    """Null function.
16257
16258    >>> nullfunc('arg', kwarg='kwarg')
16259
16260    """
16261    return
16262
16263
16264def sequence(value):
16265    """Return tuple containing value if value is not a tuple or list.
16266
16267    >>> sequence(1)
16268    (1,)
16269    >>> sequence([1])
16270    [1]
16271    >>> sequence('ab')
16272    ('ab',)
16273
16274    """
16275    return value if isinstance(value, (tuple, list)) else (value,)
16276
16277
16278def product(iterable):
16279    """Return product of sequence of numbers.
16280
16281    Equivalent of functools.reduce(operator.mul, iterable, 1).
16282    Multiplying numpy integers might overflow.
16283
16284    >>> product([2**8, 2**30])
16285    274877906944
16286    >>> product([])
16287    1
16288
16289    """
16290    prod = 1
16291    for i in iterable:
16292        prod *= i
16293    return prod
16294
16295
16296def natural_sorted(iterable):
16297    """Return human sorted list of strings.
16298
16299    E.g. for sorting file names.
16300
16301    >>> natural_sorted(['f1', 'f2', 'f10'])
16302    ['f1', 'f2', 'f10']
16303
16304    """
16305
16306    def sortkey(x):
16307        return [(int(c) if c.isdigit() else c) for c in re.split(numbers, x)]
16308
16309    numbers = re.compile(r'(\d+)')
16310    return sorted(iterable, key=sortkey)
16311
16312
16313def epics_datetime(sec, nsec):
16314    """Return datetime object from epicsTSSec and epicsTSNsec tag values."""
16315    return datetime.datetime.fromtimestamp(sec + 631152000 + nsec / 1e9)
16316
16317
16318def excel_datetime(timestamp, epoch=None):
16319    """Return datetime object from timestamp in Excel serial format.
16320
16321    Convert LSM time stamps.
16322
16323    >>> excel_datetime(40237.029999999795)
16324    datetime.datetime(2010, 2, 28, 0, 43, 11, 999982)
16325
16326    """
16327    if epoch is None:
16328        epoch = datetime.datetime.fromordinal(693594)
16329    return epoch + datetime.timedelta(timestamp)
16330
16331
16332def julian_datetime(julianday, milisecond=0):
16333    """Return datetime from days since 1/1/4713 BC and ms since midnight.
16334
16335    Convert Julian dates according to MetaMorph.
16336
16337    >>> julian_datetime(2451576, 54362783)
16338    datetime.datetime(2000, 2, 2, 15, 6, 2, 783)
16339
16340    """
16341    if julianday <= 1721423:
16342        # no datetime before year 1
16343        return None
16344
16345    a = julianday + 1
16346    if a > 2299160:
16347        alpha = math.trunc((a - 1867216.25) / 36524.25)
16348        a += 1 + alpha - alpha // 4
16349    b = a + (1524 if a > 1721423 else 1158)
16350    c = math.trunc((b - 122.1) / 365.25)
16351    d = math.trunc(365.25 * c)
16352    e = math.trunc((b - d) / 30.6001)
16353
16354    day = b - d - math.trunc(30.6001 * e)
16355    month = e - (1 if e < 13.5 else 13)
16356    year = c - (4716 if month > 2.5 else 4715)
16357
16358    hour, milisecond = divmod(milisecond, 1000 * 60 * 60)
16359    minute, milisecond = divmod(milisecond, 1000 * 60)
16360    second, milisecond = divmod(milisecond, 1000)
16361
16362    return datetime.datetime(
16363        year, month, day, hour, minute, second, milisecond
16364    )
16365
16366
16367def byteorder_isnative(byteorder):
16368    """Return if byteorder matches the system's byteorder.
16369
16370    >>> byteorder_isnative('=')
16371    True
16372
16373    """
16374    if byteorder in ('=', sys.byteorder):
16375        return True
16376    keys = {'big': '>', 'little': '<'}
16377    return keys.get(byteorder, byteorder) == keys[sys.byteorder]
16378
16379
16380def byteorder_compare(byteorder, byteorder2):
16381    """Return if byteorders match.
16382
16383    >>> byteorder_compare('<', '<')
16384    True
16385    >>> byteorder_compare('>', '<')
16386    False
16387
16388    """
16389    if byteorder == byteorder2 or byteorder == '|' or byteorder2 == '|':
16390        return True
16391    if byteorder == '=':
16392        byteorder = {'big': '>', 'little': '<'}[sys.byteorder]
16393    elif byteorder2 == '=':
16394        byteorder2 = {'big': '>', 'little': '<'}[sys.byteorder]
16395    return byteorder == byteorder2
16396
16397
16398def recarray2dict(recarray):
16399    """Return numpy.recarray as dict."""
16400    # TODO: subarrays
16401    result = {}
16402    for descr, value in zip(recarray.dtype.descr, recarray):
16403        name, dtype = descr[:2]
16404        if dtype[1] == 'S':
16405            value = bytes2str(stripnull(value))
16406        elif value.ndim < 2:
16407            value = value.tolist()
16408        result[name] = value
16409    return result
16410
16411
16412def xml2dict(xml, sanitize=True, prefix=None):
16413    """Return XML as dict.
16414
16415    >>> xml2dict('<?xml version="1.0" ?><root attr="name"><key>1</key></root>')
16416    {'root': {'key': 1, 'attr': 'name'}}
16417    >>> xml2dict('<level1><level2>3.5322</level2></level1>')
16418    {'level1': {'level2': 3.5322}}
16419
16420    """
16421    from xml.etree import ElementTree as etree  # delayed import
16422
16423    at = tx = ''
16424    if prefix:
16425        at, tx = prefix
16426
16427    def astype(value):
16428        # return string value as int, float, bool, or unchanged
16429        if not isinstance(value, (str, bytes)):
16430            return value
16431        for t in (int, float, asbool):
16432            try:
16433                return t(value)
16434            except Exception:
16435                pass
16436        return value
16437
16438    def etree2dict(t):
16439        # adapted from https://stackoverflow.com/a/10077069/453463
16440        key = t.tag
16441        if sanitize:
16442            key = key.rsplit('}', 1)[-1]
16443        d = {key: {} if t.attrib else None}
16444        children = list(t)
16445        if children:
16446            dd = collections.defaultdict(list)
16447            for dc in map(etree2dict, children):
16448                for k, v in dc.items():
16449                    dd[k].append(astype(v))
16450            d = {
16451                key: {
16452                    k: astype(v[0]) if len(v) == 1 else astype(v)
16453                    for k, v in dd.items()
16454                }
16455            }
16456        if t.attrib:
16457            d[key].update((at + k, astype(v)) for k, v in t.attrib.items())
16458        if t.text:
16459            text = t.text.strip()
16460            if children or t.attrib:
16461                if text:
16462                    d[key][tx + 'value'] = astype(text)
16463            else:
16464                d[key] = astype(text)
16465        return d
16466
16467    return etree2dict(etree.fromstring(xml))
16468
16469
16470def hexdump(bytestr, width=75, height=24, snipat=-2, modulo=2, ellipsis=None):
16471    """Return hexdump representation of bytes.
16472
16473    >>> hexdump(binascii.unhexlify('49492a00080000000e00fe0004000100'))
16474    '49 49 2a 00 08 00 00 00 0e 00 fe 00 04 00 01 00 II*.............'
16475
16476    """
16477    size = len(bytestr)
16478    if size < 1 or width < 2 or height < 1:
16479        return ''
16480    if height == 1:
16481        addr = b''
16482        bytesperline = min(
16483            modulo * (((width - len(addr)) // 4) // modulo), size
16484        )
16485        if bytesperline < 1:
16486            return ''
16487        nlines = 1
16488    else:
16489        addr = b'%%0%ix: ' % len(b'%x' % size)
16490        bytesperline = min(
16491            modulo * (((width - len(addr % 1)) // 4) // modulo), size
16492        )
16493        if bytesperline < 1:
16494            return ''
16495        width = 3 * bytesperline + len(addr % 1)
16496        nlines = (size - 1) // bytesperline + 1
16497
16498    if snipat is None or snipat == 1:
16499        snipat = height
16500    elif 0 < abs(snipat) < 1:
16501        snipat = int(math.floor(height * snipat))
16502    if snipat < 0:
16503        snipat += height
16504
16505    if height == 1 or nlines == 1:
16506        blocks = [(0, bytestr[:bytesperline])]
16507        addr = b''
16508        height = 1
16509        width = 3 * bytesperline
16510    elif height is None or nlines <= height:
16511        blocks = [(0, bytestr)]
16512    elif snipat <= 0:
16513        start = bytesperline * (nlines - height)
16514        blocks = [(start, bytestr[start:])]  # (start, None)
16515    elif snipat >= height or height < 3:
16516        end = bytesperline * height
16517        blocks = [(0, bytestr[:end])]  # (end, None)
16518    else:
16519        end1 = bytesperline * snipat
16520        end2 = bytesperline * (height - snipat - 1)
16521        blocks = [
16522            (0, bytestr[:end1]),
16523            (size - end1 - end2, None),
16524            (size - end2, bytestr[size - end2 :]),
16525        ]
16526
16527    ellipsis = b'...' if ellipsis is None else ellipsis.encode('cp1252')
16528    result = []
16529    for start, bytestr in blocks:
16530        if bytestr is None:
16531            result.append(ellipsis)  # 'skip %i bytes' % start)
16532            continue
16533        hexstr = binascii.hexlify(bytestr)
16534        strstr = re.sub(br'[^\x20-\x7f]', b'.', bytestr)
16535        for i in range(0, len(bytestr), bytesperline):
16536            h = hexstr[2 * i : 2 * i + bytesperline * 2]
16537            r = (addr % (i + start)) if height > 1 else addr
16538            r += b' '.join(h[i : i + 2] for i in range(0, 2 * bytesperline, 2))
16539            r += b' ' * (width - len(r))
16540            r += strstr[i : i + bytesperline]
16541            result.append(r)
16542    result = b'\n'.join(result)
16543    result = result.decode('ascii')
16544    return result
16545
16546
16547def isprintable(string):
16548    r"""Return if all characters in string are printable.
16549
16550    >>> isprintable('abc')
16551    True
16552    >>> isprintable(b'\01')
16553    False
16554
16555    """
16556    string = string.strip()
16557    if not string:
16558        return True
16559    try:
16560        return string.isprintable()
16561    except Exception:
16562        pass
16563    try:
16564        return string.decode().isprintable()
16565    except Exception:
16566        pass
16567
16568
16569def clean_whitespace(string, compact=False):
16570    """Return string with compressed whitespace."""
16571    for a, b in (
16572        ('\r\n', '\n'),
16573        ('\r', '\n'),
16574        ('\n\n', '\n'),
16575        ('\t', ' '),
16576        ('  ', ' '),
16577    ):
16578        string = string.replace(a, b)
16579    if compact:
16580        for a, b in (
16581            ('\n', ' '),
16582            ('[ ', '['),
16583            ('  ', ' '),
16584            ('  ', ' '),
16585            ('  ', ' '),
16586        ):
16587            string = string.replace(a, b)
16588    return string.strip()
16589
16590
16591def pformat_xml(xml):
16592    """Return pretty formatted XML."""
16593    try:
16594        from lxml import etree  # delayed import
16595
16596        if not isinstance(xml, bytes):
16597            xml = xml.encode()
16598        tree = etree.parse(io.BytesIO(xml))
16599        xml = etree.tostring(
16600            tree,
16601            pretty_print=True,
16602            xml_declaration=True,
16603            encoding=tree.docinfo.encoding,
16604        )
16605        xml = bytes2str(xml)
16606    except Exception:
16607        if isinstance(xml, bytes):
16608            xml = bytes2str(xml)
16609        xml = xml.replace('><', '>\n<')
16610    return xml.replace('  ', ' ').replace('\t', ' ')
16611
16612
16613def pformat(arg, width=79, height=24, compact=True):
16614    """Return pretty formatted representation of object as string.
16615
16616    Whitespace might be altered.
16617
16618    """
16619    if height is None or height < 1:
16620        height = 1024
16621    if width is None or width < 1:
16622        width = 256
16623
16624    npopt = numpy.get_printoptions()
16625    numpy.set_printoptions(threshold=100, linewidth=width)
16626
16627    if isinstance(arg, bytes):
16628        if arg[:5].lower() == b'<?xml' or arg[-4:] == b'OME>':
16629            arg = bytes2str(arg)
16630
16631    if isinstance(arg, bytes):
16632        if isprintable(arg):
16633            arg = bytes2str(arg)
16634            arg = clean_whitespace(arg)
16635        else:
16636            numpy.set_printoptions(**npopt)
16637            return hexdump(arg, width=width, height=height, modulo=1)
16638        arg = arg.rstrip()
16639    elif isinstance(arg, str):
16640        if arg[:5].lower() == '<?xml' or arg[-4:] == 'OME>':
16641            arg = arg[: 4 * width] if height == 1 else pformat_xml(arg)
16642        arg = arg.rstrip()
16643    elif isinstance(arg, numpy.record):
16644        arg = arg.pprint()
16645    else:
16646        import pprint  # delayed import
16647
16648        arg = pprint.pformat(arg, width=width, compact=compact)
16649
16650    numpy.set_printoptions(**npopt)
16651
16652    if height == 1:
16653        arg = clean_whitespace(arg, compact=True)
16654        return arg[:width]
16655
16656    argl = list(arg.splitlines())
16657    if len(argl) > height:
16658        arg = '\n'.join(argl[: height // 2] + ['...'] + argl[-height // 2 :])
16659    return arg
16660
16661
16662def snipstr(string, width=79, snipat=None, ellipsis=None):
16663    """Return string cut to specified length.
16664
16665    >>> snipstr('abcdefghijklmnop', 8)
16666    'abc...op'
16667
16668    """
16669    if snipat is None:
16670        snipat = 0.5
16671    if ellipsis is None:
16672        if isinstance(string, bytes):
16673            ellipsis = b'...'
16674        else:
16675            ellipsis = '\u2026'
16676    esize = len(ellipsis)
16677
16678    splitlines = string.splitlines()
16679    # TODO: finish and test multiline snip
16680
16681    result = []
16682    for line in splitlines:
16683        if line is None:
16684            result.append(ellipsis)
16685            continue
16686        linelen = len(line)
16687        if linelen <= width:
16688            result.append(string)
16689            continue
16690
16691        split = snipat
16692        if split is None or split == 1:
16693            split = linelen
16694        elif 0 < abs(split) < 1:
16695            split = int(math.floor(linelen * split))
16696        if split < 0:
16697            split += linelen
16698            if split < 0:
16699                split = 0
16700
16701        if esize == 0 or width < esize + 1:
16702            if split <= 0:
16703                result.append(string[-width:])
16704            else:
16705                result.append(string[:width])
16706        elif split <= 0:
16707            result.append(ellipsis + string[esize - width :])
16708        elif split >= linelen or width < esize + 4:
16709            result.append(string[: width - esize] + ellipsis)
16710        else:
16711            splitlen = linelen - width + esize
16712            end1 = split - splitlen // 2
16713            end2 = end1 + splitlen
16714            result.append(string[:end1] + ellipsis + string[end2:])
16715
16716    if isinstance(string, bytes):
16717        return b'\n'.join(result)
16718    return '\n'.join(result)
16719
16720
16721def enumstr(enum):
16722    """Return short string representation of Enum instance."""
16723    name = enum.name
16724    if name is None:
16725        name = str(enum)
16726    return name
16727
16728
16729def enumarg(enum, arg):
16730    """Return enum member from its name or value.
16731
16732    >>> enumarg(TIFF.PHOTOMETRIC, 2)
16733    <PHOTOMETRIC.RGB: 2>
16734    >>> enumarg(TIFF.PHOTOMETRIC, 'RGB')
16735    <PHOTOMETRIC.RGB: 2>
16736
16737    """
16738    try:
16739        return enum(arg)
16740    except Exception:
16741        try:
16742            return enum[arg.upper()]
16743        except Exception:
16744            raise ValueError(f'invalid argument {arg}')
16745
16746
16747def parse_kwargs(kwargs, *keys, **keyvalues):
16748    """Return dict with keys from keys|keyvals and values from kwargs|keyvals.
16749
16750    Existing keys are deleted from kwargs.
16751
16752    >>> kwargs = {'one': 1, 'two': 2, 'four': 4}
16753    >>> kwargs2 = parse_kwargs(kwargs, 'two', 'three', four=None, five=5)
16754    >>> kwargs == {'one': 1}
16755    True
16756    >>> kwargs2 == {'two': 2, 'four': 4, 'five': 5}
16757    True
16758
16759    """
16760    result = {}
16761    for key in keys:
16762        if key in kwargs:
16763            result[key] = kwargs[key]
16764            del kwargs[key]
16765    for key, value in keyvalues.items():
16766        if key in kwargs:
16767            result[key] = kwargs[key]
16768            del kwargs[key]
16769        else:
16770            result[key] = value
16771    return result
16772
16773
16774def update_kwargs(kwargs, **keyvalues):
16775    """Update dict with keys and values if keys do not already exist.
16776
16777    >>> kwargs = {'one': 1, }
16778    >>> update_kwargs(kwargs, one=None, two=2)
16779    >>> kwargs == {'one': 1, 'two': 2}
16780    True
16781
16782    """
16783    for key, value in keyvalues.items():
16784        if key not in kwargs:
16785            kwargs[key] = value
16786
16787
16788def log_warning(msg, *args, **kwargs):
16789    """Log message with level WARNING."""
16790    import logging
16791
16792    logging.getLogger(__name__).warning(msg, *args, **kwargs)
16793
16794
16795def validate_jhove(filename, jhove=None, ignore=None):
16796    """Validate TIFF file using jhove -m TIFF-hul.
16797
16798    Raise ValueError if jhove outputs an error message unless the message
16799    contains one of the strings in 'ignore'.
16800
16801    JHOVE does not support bigtiff or more than 50 IFDs.
16802
16803    See `JHOVE TIFF-hul Module <http://jhove.sourceforge.net/tiff-hul.html>`_
16804
16805    """
16806    import subprocess
16807
16808    if ignore is None:
16809        ignore = ['More than 50 IFDs']
16810    if jhove is None:
16811        jhove = 'jhove'
16812    out = subprocess.check_output([jhove, filename, '-m', 'TIFF-hul'])
16813    if b'ErrorMessage: ' in out:
16814        for line in out.splitlines():
16815            line = line.strip()
16816            if line.startswith(b'ErrorMessage: '):
16817                error = line[14:].decode()
16818                for i in ignore:
16819                    if i in error:
16820                        break
16821                else:
16822                    raise ValueError(error)
16823                break
16824
16825
16826def tiffcomment(arg, comment=None, index=None, code=None):
16827    """Return or replace ImageDescription value in first page of TIFF file."""
16828    if index is None:
16829        index = 0
16830    if code is None:
16831        code = 270
16832    mode = None if comment is None else 'r+b'
16833    with TiffFile(arg, mode=mode) as tif:
16834        tag = tif.pages[index].tags.get(code, None)
16835        if tag is None:
16836            raise ValueError(f'no {TIFF.TAGS[code]} tag found')
16837        if comment is None:
16838            return tag.value
16839        tag.overwrite(comment)
16840
16841
16842def tiff2fsspec(
16843    filename,
16844    url,
16845    out=None,
16846    key=None,
16847    series=None,
16848    level=None,
16849    chunkmode=None,
16850    version=None,
16851):
16852    """Write fsspec ReferenceFileSystem JSON from TIFF file."""
16853    if out is None:
16854        out = filename + '.json'
16855    with TiffFile(filename) as tif:
16856        with tif.aszarr(
16857            key=key, series=series, level=level, chunkmode=chunkmode
16858        ) as store:
16859            store.write_fsspec(out, url, version=version)
16860
16861
16862def lsm2bin(lsmfile, binfile=None, tile=None, verbose=True):
16863    """Convert [MP]TZCYX LSM file to series of BIN files.
16864
16865    One BIN file containing 'ZCYX' data are created for each position, time,
16866    and tile. The position, time, and tile indices are encoded at the end
16867    of the filenames.
16868
16869    """
16870    verbose = print if verbose else nullfunc
16871
16872    if tile is None:
16873        tile = (256, 256)
16874
16875    if binfile is None:
16876        binfile = lsmfile
16877    elif binfile.lower() == 'none':
16878        binfile = None
16879    if binfile:
16880        binfile += '_(z%ic%iy%ix%i)_m%%ip%%it%%03iy%%ix%%i.bin'
16881
16882    verbose('\nOpening LSM file... ', end='', flush=True)
16883    timer = Timer()
16884
16885    with TiffFile(lsmfile) as lsm:
16886        if not lsm.is_lsm:
16887            verbose('\n', lsm, flush=True)
16888            raise ValueError('not a LSM file')
16889        series = lsm.series[0]  # first series contains the image data
16890        shape = series.get_shape(False)
16891        axes = series.get_axes(False)
16892        dtype = series.dtype
16893        size = product(shape) * dtype.itemsize
16894
16895        verbose(timer)
16896        # verbose(lsm, flush=True)
16897        verbose(
16898            'Image\n  axes:  {}\n  shape: {}\n  dtype: {}\n  size:  {}'.format(
16899                axes, shape, dtype, format_size(size)
16900            ),
16901            flush=True,
16902        )
16903        if not series.axes.endswith('TZCYX'):
16904            raise ValueError('not a *TZCYX LSM file')
16905
16906        verbose('Copying image from LSM to BIN files', end='', flush=True)
16907        timer.start()
16908        tiles = shape[-2] // tile[-2], shape[-1] // tile[-1]
16909        if binfile:
16910            binfile = binfile % (shape[-4], shape[-3], tile[0], tile[1])
16911        shape = (1,) * (7 - len(shape)) + shape
16912        # cache for ZCYX stacks and output files
16913        data = numpy.empty(shape[3:], dtype=dtype)
16914        out = numpy.empty(
16915            (shape[-4], shape[-3], tile[0], tile[1]), dtype=dtype
16916        )
16917        # iterate over Tiff pages containing data
16918        pages = iter(series.pages)
16919        for m in range(shape[0]):  # mosaic axis
16920            for p in range(shape[1]):  # position axis
16921                for t in range(shape[2]):  # time axis
16922                    for z in range(shape[3]):  # z slices
16923                        data[z] = next(pages).asarray()
16924                    for y in range(tiles[0]):  # tile y
16925                        for x in range(tiles[1]):  # tile x
16926                            out[:] = data[
16927                                ...,
16928                                y * tile[0] : (y + 1) * tile[0],
16929                                x * tile[1] : (x + 1) * tile[1],
16930                            ]
16931                            if binfile:
16932                                out.tofile(binfile % (m, p, t, y, x))
16933                            verbose('.', end='', flush=True)
16934        verbose(timer, flush=True)
16935
16936
16937def imshow(
16938    data,
16939    photometric=None,
16940    planarconfig=None,
16941    bitspersample=None,
16942    nodata=0,
16943    interpolation=None,
16944    cmap=None,
16945    vmin=None,
16946    vmax=None,
16947    figure=None,
16948    title=None,
16949    dpi=96,
16950    subplot=None,
16951    maxdim=None,
16952    **kwargs,
16953):
16954    """Plot n-dimensional images using matplotlib.pyplot.
16955
16956    Return figure, subplot, and plot axis.
16957    Requires pyplot already imported C{from matplotlib import pyplot}.
16958
16959    Parameters
16960    ----------
16961    data : nd array
16962        The image data.
16963    photometric : {'MINISWHITE', 'MINISBLACK', 'RGB', or 'PALETTE'}
16964        The color space of the image data.
16965    planarconfig : {'CONTIG' or 'SEPARATE'}
16966        Defines how components of each pixel are stored.
16967    bitspersample : int
16968        Number of bits per channel in integer RGB images.
16969    interpolation : str
16970        The image interpolation method used in matplotlib.imshow. By default,
16971        'nearest' is used for image dimensions <= 512, else 'bilinear'.
16972    cmap : str or matplotlib.colors.Colormap
16973        The colormap maps non-RGBA scalar data to colors.
16974    vmin, vmax : scalar
16975        Data range covered by the colormap. By default, the complete
16976        range of the data is covered.
16977    figure : matplotlib.figure.Figure
16978        Matplotlib figure to use for plotting.
16979    title : str
16980        Window and subplot title.
16981    subplot : int
16982        A matplotlib.pyplot.subplot axis.
16983    maxdim : int
16984        Maximum image width and length.
16985    kwargs : dict
16986        Additional arguments for matplotlib.pyplot.imshow.
16987
16988    """
16989    # TODO: rewrite detection of isrgb, iscontig
16990    # TODO: use planarconfig
16991    if photometric is None:
16992        photometric = 'RGB'
16993    if maxdim is None:
16994        maxdim = 2 ** 16
16995    isrgb = photometric in ('RGB', 'YCBCR')  # 'PALETTE', 'YCBCR'
16996
16997    if data.dtype == 'float16':
16998        data = data.astype('float32')
16999
17000    if data.dtype.kind == 'b':
17001        isrgb = False
17002
17003    if isrgb and not (
17004        data.shape[-1] in (3, 4)
17005        or (data.ndim > 2 and data.shape[-3] in (3, 4))
17006    ):
17007        isrgb = False
17008        photometric = 'MINISBLACK'
17009
17010    data = data.squeeze()
17011    if photometric in ('MINISWHITE', 'MINISBLACK', None):
17012        data = reshape_nd(data, 2)
17013    else:
17014        data = reshape_nd(data, 3)
17015
17016    dims = data.ndim
17017    if dims < 2:
17018        raise ValueError('not an image')
17019    if dims == 2:
17020        dims = 0
17021        isrgb = False
17022    else:
17023        if isrgb and data.shape[-3] in (3, 4):
17024            data = numpy.swapaxes(data, -3, -2)
17025            data = numpy.swapaxes(data, -2, -1)
17026        elif not isrgb and (
17027            data.shape[-1] < data.shape[-2] // 8
17028            and data.shape[-1] < data.shape[-3] // 8
17029        ):
17030            data = numpy.swapaxes(data, -3, -1)
17031            data = numpy.swapaxes(data, -2, -1)
17032        isrgb = isrgb and data.shape[-1] in (3, 4)
17033        dims -= 3 if isrgb else 2
17034
17035    if interpolation is None:
17036        threshold = 512
17037    elif isinstance(interpolation, int):
17038        threshold = interpolation
17039    else:
17040        threshold = 0
17041
17042    if isrgb:
17043        data = data[..., :maxdim, :maxdim, :maxdim]
17044        if threshold:
17045            if data.shape[-2] > threshold or data.shape[-3] > threshold:
17046                interpolation = 'bilinear'
17047            else:
17048                interpolation = 'nearest'
17049    else:
17050        data = data[..., :maxdim, :maxdim]
17051        if threshold:
17052            if data.shape[-1] > threshold or data.shape[-2] > threshold:
17053                interpolation = 'bilinear'
17054            else:
17055                interpolation = 'nearest'
17056
17057    if photometric == 'PALETTE' and isrgb:
17058        try:
17059            datamax = numpy.max(data)
17060        except ValueError:
17061            datamax = 1
17062        if datamax > 255:
17063            data = data >> 8  # possible precision loss
17064        data = data.astype('B')
17065    elif data.dtype.kind in 'ui':
17066        if not (isrgb and data.dtype.itemsize <= 1) or bitspersample is None:
17067            try:
17068                bitspersample = int(math.ceil(math.log(data.max(), 2)))
17069            except Exception:
17070                bitspersample = data.dtype.itemsize * 8
17071        elif not isinstance(bitspersample, (int, numpy.integer)):
17072            # bitspersample can be tuple, e.g. (5, 6, 5)
17073            bitspersample = data.dtype.itemsize * 8
17074        datamax = 2 ** bitspersample
17075        if isrgb:
17076            if bitspersample < 8:
17077                data = data << (8 - bitspersample)
17078            elif bitspersample > 8:
17079                data = data >> (bitspersample - 8)  # precision loss
17080            data = data.astype('B')
17081    elif data.dtype.kind == 'f':
17082        if nodata:
17083            data = data.copy()
17084            data[data > 1e30] = 0.0
17085        try:
17086            datamax = numpy.max(data)
17087        except ValueError:
17088            datamax = 1
17089        if isrgb and datamax > 1.0:
17090            if data.dtype.char == 'd':
17091                data = data.astype('f')
17092                data /= datamax
17093            else:
17094                data = data / datamax
17095    elif data.dtype.kind == 'b':
17096        datamax = 1
17097    elif data.dtype.kind == 'c':
17098        data = numpy.absolute(data)
17099        try:
17100            datamax = numpy.max(data)
17101        except ValueError:
17102            datamax = 1
17103
17104    if isrgb:
17105        vmin = 0
17106    else:
17107        if vmax is None:
17108            vmax = datamax
17109        if vmin is None:
17110            if data.dtype.kind == 'i':
17111                dtmin = numpy.iinfo(data.dtype).min
17112                try:
17113                    vmin = numpy.min(data)
17114                except ValueError:
17115                    vmin = -1
17116                if vmin == dtmin:
17117                    vmin = numpy.min(data[data > dtmin])
17118            elif data.dtype.kind == 'f':
17119                dtmin = numpy.finfo(data.dtype).min
17120                try:
17121                    vmin = numpy.min(data)
17122                except ValueError:
17123                    vmin = 0.0
17124                if vmin == dtmin:
17125                    vmin = numpy.min(data[data > dtmin])
17126            else:
17127                vmin = 0
17128
17129    pyplot = sys.modules['matplotlib.pyplot']
17130
17131    if figure is None:
17132        pyplot.rc('font', family='sans-serif', weight='normal', size=8)
17133        figure = pyplot.figure(
17134            dpi=dpi,
17135            figsize=(10.3, 6.3),
17136            frameon=True,
17137            facecolor='1.0',
17138            edgecolor='w',
17139        )
17140        try:
17141            figure.canvas.manager.window.title(title)
17142        except Exception:
17143            pass
17144        size = len(title.splitlines()) if title else 1
17145        pyplot.subplots_adjust(
17146            bottom=0.03 * (dims + 2),
17147            top=0.98 - size * 0.03,
17148            left=0.1,
17149            right=0.95,
17150            hspace=0.05,
17151            wspace=0.0,
17152        )
17153    if subplot is None:
17154        subplot = 111
17155    subplot = pyplot.subplot(subplot)
17156    subplot.set_facecolor((0, 0, 0))
17157
17158    if title:
17159        try:
17160            title = str(title, 'Windows-1252')
17161        except TypeError:
17162            pass
17163        pyplot.title(title, size=11)
17164
17165    if cmap is None:
17166        if data.dtype.char == '?':
17167            cmap = 'gray'
17168        elif data.dtype.kind in 'buf' or vmin == 0:
17169            cmap = 'viridis'
17170        else:
17171            cmap = 'coolwarm'
17172        if photometric == 'MINISWHITE':
17173            cmap += '_r'
17174
17175    image = pyplot.imshow(
17176        numpy.atleast_2d(data[(0,) * dims].squeeze()),
17177        vmin=vmin,
17178        vmax=vmax,
17179        cmap=cmap,
17180        interpolation=interpolation,
17181        **kwargs,
17182    )
17183
17184    if not isrgb:
17185        pyplot.colorbar()  # panchor=(0.55, 0.5), fraction=0.05
17186
17187    def format_coord(x, y):
17188        # callback function to format coordinate display in toolbar
17189        x = int(x + 0.5)
17190        y = int(y + 0.5)
17191        try:
17192            if dims:
17193                return f'{curaxdat[1][y, x]} @ {current} [{y:4}, {x:4}]'
17194            return f'{data[y, x]} @ [{y:4}, {x:4}]'
17195        except IndexError:
17196            return ''
17197
17198    def none(event):
17199        return ''
17200
17201    subplot.format_coord = format_coord
17202    image.get_cursor_data = none
17203    image.format_cursor_data = none
17204
17205    if dims:
17206        current = list((0,) * dims)
17207        curaxdat = [0, data[tuple(current)].squeeze()]
17208        sliders = [
17209            pyplot.Slider(
17210                pyplot.axes([0.125, 0.03 * (axis + 1), 0.725, 0.025]),
17211                f'Dimension {axis}',
17212                0,
17213                data.shape[axis] - 1,
17214                0,
17215                facecolor='0.5',
17216                valfmt=f'%.0f [{data.shape[axis]}]',
17217            )
17218            for axis in range(dims)
17219        ]
17220        for slider in sliders:
17221            slider.drawon = False
17222
17223        def set_image(current, sliders=sliders, data=data):
17224            # change image and redraw canvas
17225            curaxdat[1] = data[tuple(current)].squeeze()
17226            image.set_data(curaxdat[1])
17227            for ctrl, index in zip(sliders, current):
17228                ctrl.eventson = False
17229                ctrl.set_val(index)
17230                ctrl.eventson = True
17231            figure.canvas.draw()
17232
17233        def on_changed(index, axis, data=data, current=current):
17234            # callback function for slider change event
17235            index = int(round(index))
17236            curaxdat[0] = axis
17237            if index == current[axis]:
17238                return
17239            if index >= data.shape[axis]:
17240                index = 0
17241            elif index < 0:
17242                index = data.shape[axis] - 1
17243            current[axis] = index
17244            set_image(current)
17245
17246        def on_keypressed(event, data=data, current=current):
17247            # callback function for key press event
17248            key = event.key
17249            axis = curaxdat[0]
17250            if str(key) in '0123456789':
17251                on_changed(key, axis)
17252            elif key == 'right':
17253                on_changed(current[axis] + 1, axis)
17254            elif key == 'left':
17255                on_changed(current[axis] - 1, axis)
17256            elif key == 'up':
17257                curaxdat[0] = 0 if axis == len(data.shape) - 1 else axis + 1
17258            elif key == 'down':
17259                curaxdat[0] = len(data.shape) - 1 if axis == 0 else axis - 1
17260            elif key == 'end':
17261                on_changed(data.shape[axis] - 1, axis)
17262            elif key == 'home':
17263                on_changed(0, axis)
17264
17265        figure.canvas.mpl_connect('key_press_event', on_keypressed)
17266        for axis, ctrl in enumerate(sliders):
17267            ctrl.on_changed(lambda k, a=axis: on_changed(k, a))
17268
17269    return figure, subplot, image
17270
17271
17272def _app_show():
17273    """Block the GUI. For use as skimage plugin."""
17274    pyplot = sys.modules['matplotlib.pyplot']
17275    pyplot.show()
17276
17277
17278def askopenfilename(**kwargs):
17279    """Return file name(s) from Tkinter's file open dialog."""
17280    from tkinter import Tk, filedialog
17281
17282    root = Tk()
17283    root.withdraw()
17284    root.update()
17285    filenames = filedialog.askopenfilename(**kwargs)
17286    root.destroy()
17287    return filenames
17288
17289
17290def main():
17291    """Tifffile command line usage main function."""
17292    import logging
17293    import optparse  # TODO: use argparse
17294
17295    logging.getLogger(__name__).setLevel(logging.INFO)
17296
17297    parser = optparse.OptionParser(
17298        usage='usage: %prog [options] path',
17299        description='Display image data in TIFF files.',
17300        version=f'%prog {__version__}',
17301        prog='tifffile',
17302    )
17303    opt = parser.add_option
17304    opt(
17305        '-p',
17306        '--page',
17307        dest='page',
17308        type='int',
17309        default=-1,
17310        help='display single page',
17311    )
17312    opt(
17313        '-s',
17314        '--series',
17315        dest='series',
17316        type='int',
17317        default=-1,
17318        help='display series of pages of same shape',
17319    )
17320    opt(
17321        '-l',
17322        '--level',
17323        dest='level',
17324        type='int',
17325        default=-1,
17326        help='display pyramid level of series',
17327    )
17328    opt(
17329        '--nomultifile',
17330        dest='nomultifile',
17331        action='store_true',
17332        default=False,
17333        help='do not read OME series from multiple files',
17334    )
17335    opt(
17336        '--noplots',
17337        dest='noplots',
17338        type='int',
17339        default=10,
17340        help='maximum number of plots',
17341    )
17342    opt(
17343        '--interpol',
17344        dest='interpol',
17345        metavar='INTERPOL',
17346        default=None,
17347        help='image interpolation method',
17348    )
17349    opt('--dpi', dest='dpi', type='int', default=96, help='plot resolution')
17350    opt(
17351        '--vmin',
17352        dest='vmin',
17353        type='int',
17354        default=None,
17355        help='minimum value for colormapping',
17356    )
17357    opt(
17358        '--vmax',
17359        dest='vmax',
17360        type='int',
17361        default=None,
17362        help='maximum value for colormapping',
17363    )
17364    opt(
17365        '--debug',
17366        dest='debug',
17367        action='store_true',
17368        default=False,
17369        help='raise exception on failures',
17370    )
17371    opt(
17372        '--doctest',
17373        dest='doctest',
17374        action='store_true',
17375        default=False,
17376        help='runs the docstring examples',
17377    )
17378    opt('-v', '--detail', dest='detail', type='int', default=2)
17379    opt('-q', '--quiet', dest='quiet', action='store_true')
17380
17381    settings, path = parser.parse_args()
17382    path = ' '.join(path)
17383
17384    if settings.doctest:
17385        import doctest
17386
17387        try:
17388            import tifffile.tifffile as m
17389        except ImportError:
17390            m = None
17391        doctest.testmod(m, optionflags=doctest.ELLIPSIS)
17392        return 0
17393    if not path:
17394        path = askopenfilename(
17395            title='Select a TIFF file', filetypes=TIFF.FILEOPEN_FILTER
17396        )
17397        if not path:
17398            parser.error('No file specified')
17399
17400    if any(i in path for i in '?*'):
17401        path = glob.glob(path)
17402        if not path:
17403            print('No files match the pattern')
17404            return 0
17405        # TODO: handle image sequences
17406        path = path[0]
17407
17408    if not settings.quiet:
17409        print('\nReading TIFF header:', end=' ', flush=True)
17410    timer = Timer()
17411    try:
17412        tif = TiffFile(path, _multifile=not settings.nomultifile)
17413    except Exception as exc:
17414        if settings.debug:
17415            raise
17416        print(f'\n\n{exc.__class__.__name__}: {exc}')
17417        sys.exit(0)
17418
17419    if not settings.quiet:
17420        print(timer)
17421
17422    if tif.is_ome:
17423        settings.norgb = True
17424
17425    images = []
17426    if settings.noplots > 0:
17427        if not settings.quiet:
17428            print('Reading image data: ', end=' ', flush=True)
17429
17430        def notnone(x):
17431            return next(i for i in x if i is not None)
17432
17433        timer.start()
17434        try:
17435            if settings.page >= 0:
17436                images = [
17437                    (
17438                        tif.asarray(key=settings.page),
17439                        tif.pages[settings.page],
17440                        None,
17441                    )
17442                ]
17443            elif settings.series >= 0:
17444                series = tif.series[settings.series]
17445                if settings.level >= 0:
17446                    level = settings.level
17447                elif series.is_pyramidal and product(series.shape) > 2 ** 32:
17448                    level = -1
17449                    for r in series.levels:
17450                        level += 1
17451                        if product(r.shape) < 2 ** 32:
17452                            break
17453                else:
17454                    level = 0
17455                images = [
17456                    (
17457                        tif.asarray(series=settings.series, level=level),
17458                        notnone(tif.series[settings.series]._pages),
17459                        tif.series[settings.series],
17460                    )
17461                ]
17462            else:
17463                for i, s in enumerate(tif.series[: settings.noplots]):
17464                    if settings.level < 0:
17465                        level = -1
17466                        for r in s.levels:
17467                            level += 1
17468                            if product(r.shape) < 2 ** 31:
17469                                break
17470                    else:
17471                        level = 0
17472                    try:
17473                        images.append(
17474                            (
17475                                tif.asarray(series=i, level=level),
17476                                notnone(s._pages),
17477                                tif.series[i],
17478                            )
17479                        )
17480                    except Exception as exc:
17481                        images.append((None, notnone(s.pages), None))
17482                        if settings.debug:
17483                            raise
17484                        print(
17485                            '\nSeries {} failed with {}: {}... '.format(
17486                                i, exc.__class__.__name__, exc
17487                            ),
17488                            end='',
17489                        )
17490        except Exception as exc:
17491            if settings.debug:
17492                raise
17493            print(f'{exc.__class__.__name__}: {exc}')
17494
17495        if not settings.quiet:
17496            print(timer)
17497
17498    if not settings.quiet:
17499        print('Generating report:', end='   ', flush=True)
17500        timer.start()
17501        info = TiffFile.__str__(tif, detail=int(settings.detail))
17502        print(timer)
17503        print()
17504        print(info)
17505        print()
17506    tif.close()
17507
17508    if images and settings.noplots > 0:
17509        try:
17510            import matplotlib
17511
17512            matplotlib.use('TkAgg')
17513            from matplotlib import pyplot
17514        except ImportError as exc:
17515            log_warning(f'tifffile.main: {exc.__class__.__name__}: {exc}')
17516        else:
17517            for img, page, series in images:
17518                if img is None:
17519                    continue
17520                keyframe = page.keyframe
17521                vmin, vmax = settings.vmin, settings.vmax
17522                if keyframe.nodata:
17523                    try:
17524                        vmin = numpy.min(img[img > keyframe.nodata])
17525                    except ValueError:
17526                        pass
17527                if tif.is_stk:
17528                    try:
17529                        vmin = tif.stk_metadata['MinScale']
17530                        vmax = tif.stk_metadata['MaxScale']
17531                    except KeyError:
17532                        pass
17533                    else:
17534                        if vmax <= vmin:
17535                            vmin, vmax = settings.vmin, settings.vmax
17536                if series:
17537                    title = f'{tif}\n{page}\n{series}'
17538                else:
17539                    title = f'{tif}\n {page}'
17540                photometric = 'MINISBLACK'
17541                if keyframe.photometric not in (3,):
17542                    photometric = TIFF.PHOTOMETRIC(keyframe.photometric).name
17543                imshow(
17544                    img,
17545                    title=title,
17546                    vmin=vmin,
17547                    vmax=vmax,
17548                    bitspersample=keyframe.bitspersample,
17549                    nodata=keyframe.nodata,
17550                    photometric=photometric,
17551                    interpolation=settings.interpol,
17552                    dpi=settings.dpi,
17553                )
17554            pyplot.show()
17555    return 0
17556
17557
17558def bytes2str(b, encoding=None, errors='strict'):
17559    """Return Unicode string from encoded bytes."""
17560    if encoding is not None:
17561        return b.decode(encoding, errors)
17562    try:
17563        return b.decode('utf-8', errors)
17564    except UnicodeDecodeError:
17565        return b.decode('cp1252', errors)
17566
17567
17568def bytestr(s, encoding='cp1252'):
17569    """Return bytes from Unicode string, else pass through."""
17570    return s.encode(encoding) if isinstance(s, str) else s
17571
17572
17573# aliases and deprecated
17574imsave = imwrite
17575TiffWriter.save = TiffWriter.write
17576TiffReader = TiffFile
17577
17578if __name__ == '__main__':
17579    sys.exit(main())
17580