1# TileDB v2.5.2 Release Notes 2 3## Improvements 4 5* Provide non-AVX2 build artifact on Linux [#2649](https://github.com/TileDB-Inc/TileDB/pull/2649) 6* Error out when setting multiple ranges for global layout [#2658](https://github.com/TileDB-Inc/TileDB/pull/2658) 7 8## Bug fixes 9 10* Patch AWS sdk for cmake 3.22 support [#2639](https://github.com/TileDB-Inc/TileDB/pull/2639) 11* Remove assert on memory_used_result_tile_ranges_ in SparseUnorderedWithDupsReader [#2652](https://github.com/TileDB-Inc/TileDB/pull/2652) 12* Remove tiles that are empty through being filtered with a query condition [#2659](https://github.com/TileDB-Inc/TileDB/pull/2659) 13* Always load array schemas during array open to find any new array schemas created from array schema evolution [#2613](https://github.com/TileDB-Inc/TileDB/pull/2613) 14 15# TileDB v2.5.1 Release Notes 16 17## New features 18 19* Disable AVX2 for MSys2 builds used by CRAN [#2614](https://github.com/TileDB-Inc/TileDB/pull/2614) 20 21## Improvements 22 23* Clarify error messages in `check_buffers_correctness()` [#2580](https://github.com/TileDB-Inc/TileDB/pull/2580) 24 25## Bug fixes 26 27* Fix schema evolution calls on all pre-TileDB 2.4 arrays [#2611](https://github.com/TileDB-Inc/TileDB/pull/2611) 28* Unordered reads should be allowed for dense arrays [#2608](https://github.com/TileDB-Inc/TileDB/pull/2608) 29* Fix logger creation on context to be threadsafe [#2625](https://github.com/TileDB-Inc/TileDB/pull/2625) 30 31## API additions 32 33### C++ API 34 35* Add C++ API for `Context::last_error()` [#2609](https://github.com/TileDB-Inc/TileDB/pull/2609) 36 37# TileDB v2.5.0 Release Notes 38 39## Breaking C API changes 40 41* Remove deprecated c-api `tiledb_array_max_buffer_size` and `tiledb_array_max_buffer_size_var` [#2579](https://github.com/TileDB-Inc/TileDB/pull/2579) 42* Remove deprecated cpp-api `Array::max_buffer_elements` [#2579](https://github.com/TileDB-Inc/TileDB/pull/2579) 43 44## New features 45 46* Support upgrading an older version array to the latest version [#2513](https://github.com/TileDB-Inc/TileDB/pull/2513) 47* Add improved logging support to classes [#2565](https://github.com/TileDB-Inc/TileDB/pull/2565) 48 49## Improvements 50 51* Replace Buffer key_ with char key_[32] per shortcut story id 9561 [#2502](https://github.com/TileDB-Inc/TileDB/pull/2502) 52* Remove support for sparse writes in dense arrays. [#2504](https://github.com/TileDB-Inc/TileDB/pull/2504) 53* Initial dense refactor. [#2503](https://github.com/TileDB-Inc/TileDB/pull/2503) 54* More concise cmake output during build [#2512](https://github.com/TileDB-Inc/TileDB/pull/2512) 55* Sparse refactored readers: fixing looping behavior on large arrays. [#2530](https://github.com/TileDB-Inc/TileDB/pull/2530) 56* Use sparse global order reader for unordered without duplicates queries. [#2526](https://github.com/TileDB-Inc/TileDB/pull/2526) 57* Add CMakeUserPresets.json to .gitignore [#2534](https://github.com/TileDB-Inc/TileDB/pull/2534) 58* Sparse unordered with duplicates reader: support multiple ranges. [#2537](https://github.com/TileDB-Inc/TileDB/pull/2537) 59* Refactored sparse readers: tile overlap refactor. [#2547](https://github.com/TileDB-Inc/TileDB/pull/2547) 60* Refactored dense reader: fixing output buffer offsets with multi-ranges. [#2553](https://github.com/TileDB-Inc/TileDB/pull/2553) 61* Refactored sparse readers: serialization fixes. [#2558](https://github.com/TileDB-Inc/TileDB/pull/2558) 62* Refactored sparse readers: proper lifetime for tile bitmaps. [#2563](https://github.com/TileDB-Inc/TileDB/pull/2563) 63* REST scratch buffer is now owned by the query to allow reuse [#2555](https://github.com/TileDB-Inc/TileDB/pull/2555) 64* Remove default constructor from `Dimension` [#2561](https://github.com/TileDB-Inc/TileDB/pull/2561) 65* Resource pool: fixing off by one error. [#2567](https://github.com/TileDB-Inc/TileDB/pull/2567) 66* Splitting config for refactored readers. [#2569](https://github.com/TileDB-Inc/TileDB/pull/2569) 67* Sparse refactored readers: memory management unit tests. [#2568](https://github.com/TileDB-Inc/TileDB/pull/2568) 68* Removed all aspects of posix_code from Status [#2571](https://github.com/TileDB-Inc/TileDB/pull/2571) 69* Fixing pre-loading for tile offsets in various readers. [#2570](https://github.com/TileDB-Inc/TileDB/pull/2570) 70* Use the new logger in Subarray, SubarrayPartitioner and Consolidator classes. [#2574](https://github.com/TileDB-Inc/TileDB/pull/2574) 71* Add tiledb_fragment_info_get_schema_name [#2581](https://github.com/TileDB-Inc/TileDB/pull/2581) 72* Enable CMake AVX2 check [#2591](https://github.com/TileDB-Inc/TileDB/pull/2591) 73* Adding logging for sparse refactored readers. [#2575](https://github.com/TileDB-Inc/TileDB/pull/2575) 74* use ROW_MAJOR read paths for unordered reads of Hilbert layout array [#2551](https://github.com/TileDB-Inc/TileDB/pull/2551) 75 76## Bug fixes 77 78* Fix the memory leak in store_array_schema in the StorageManager class. [#2480](https://github.com/TileDB-Inc/TileDB/pull/2480) 79* Fix curl/REST query scratch size to reset after each query is processed. [#2535](https://github.com/TileDB-Inc/TileDB/pull/2535) 80* Sparse refactored readers: segfault with dimension only reads. [#2539](https://github.com/TileDB-Inc/TileDB/pull/2539) 81* REST array metadata writes should post with timestamps [#2545](https://github.com/TileDB-Inc/TileDB/pull/2545) 82* Fix bug in Arrow schema construction [#2554](https://github.com/TileDB-Inc/TileDB/pull/2554) 83* Replaced `auto& path` with `auto path` [#2560](https://github.com/TileDB-Inc/TileDB/pull/2560) 84 85## API additions 86 87### C API 88 89* Expose MBR in Fragment Info API [#2222](https://github.com/TileDB-Inc/TileDB/pull/2222) 90* Add `tiledb_fragment_info_get_array_schema_name` for fetching array name used by fragment [#2581](https://github.com/TileDB-Inc/TileDB/pull/2581) 91 92### C++ API 93 94* Expose MBR in Fragment Info API [#2222](https://github.com/TileDB-Inc/TileDB/pull/2222) 95* Add `FragmentInfo::array_schema_name` for fetching array name used by fragment [#2581](https://github.com/TileDB-Inc/TileDB/pull/2581) 96 97 98# TileDB v2.4.3 Release Notes 99 100## Bug fixes 101 102* Fix segfault in result `ResultTile::coord_string` and `ResultTile::compute_results_sparse<char>` due empty chunk buffer [#2531](https://github.com/TileDB-Inc/TileDB/pull/2531) 103* Fix memory corruption with empty result set in extra_element mode [#2540](https://github.com/TileDB-Inc/TileDB/pull/2540) 104* REST array metadata writes should post with timestamps [#2545](https://github.com/TileDB-Inc/TileDB/pull/2545) 105* Backport fixes for new Sparse Unordered with Duplicate readers from [#2530](https://github.com/TileDB-Inc/TileDB/pull/2530) and [#2538](https://github.com/TileDB-Inc/TileDB/pull/2538) 106 107## API additions 108 109# TileDB v2.4.2 Release Notes 110 111## New features 112 113* Add support for empty string as query condition value. [#2507](https://github.com/TileDB-Inc/TileDB/pull/2507) 114 115## Improvements 116 117* Support writing empty strings for dimensions [#2501](https://github.com/TileDB-Inc/TileDB/pull/2501) 118* Refactored readers can segfault when multiple contexts are used. [#2525](https://github.com/TileDB-Inc/TileDB/pull/2525) 119 120## Bug fixes 121 122* Fix ch10191: check cell_val_num for varlen status instead of result count [#2505](https://github.com/TileDB-Inc/TileDB/pull/2505) 123* Do not access variables after moving them [#2522](https://github.com/TileDB-Inc/TileDB/pull/2522) 124* Add try/catch to `tiledb_ctx_alloc` for exception safety [#2527](https://github.com/TileDB-Inc/TileDB/pull/2527) 125 126# TileDB v2.4.0 Release Notes 127 128## Disk Format 129 130* Store array schemas under `__schema` directory [#2258](https://github.com/TileDB-Inc/TileDB/pull/2258) 131 132## New features 133 134* Perform early audit for acceptable aws sdk windows path length [#2260](https://github.com/TileDB-Inc/TileDB/pull/2260) 135* Support setting via config s3 BucketCannedACL and ObjectCannedACL via SetACL() methods [#2383](https://github.com/TileDB-Inc/TileDB/pull/2383) 136* Update spdlog dependency to 1.9.0 fixing c++17 compatibility and general improvements [#1973](https://github.com/TileDB-Inc/TileDB/pull/1973) 137* Added Azure SAS token config support and new config option [#2420](https://github.com/TileDB-Inc/TileDB/pull/2420) 138* Load all array schemas in storage manager and pass the appropriate schema pointer to each fragment [#2415](https://github.com/TileDB-Inc/TileDB/pull/2415) 139* First revision of the Interval class [#2417](https://github.com/TileDB-Inc/TileDB/pull/2417) 140* Add `tiledb_schema_evolution_t` and new apis for schema evolution [#2426](https://github.com/TileDB-Inc/TileDB/pull/2426) 141* Add `ArraySchemaEvolution` to cpp_api and its unit tests are also added. [#2462](https://github.com/TileDB-Inc/TileDB/pull/2462) 142* Add c and cpp api functions for getting the array schema of a fragment [#2468](https://github.com/TileDB-Inc/TileDB/pull/2468) 143* Add capnp serialization and rest support for array schema evolution objects [#2467](https://github.com/TileDB-Inc/TileDB/pull/2467) 144 145## Improvements 146 147* `encryption_key` and `encryption_type` parameters have been added to the config; internal APIs now use these parameters to set the key. [#2245](https://github.com/TileDB-Inc/TileDB/pull/2245) 148* Initial read refactor [#2374](https://github.com/TileDB-Inc/TileDB/pull/2374) 149* Create class ByteVecValue from typedef [#2368](https://github.com/TileDB-Inc/TileDB/pull/2368) 150* Encapsulate spdlog.h [#2396](https://github.com/TileDB-Inc/TileDB/pull/2396) 151* Update OSX target to 10.14 for release artifacts [#2401](https://github.com/TileDB-Inc/TileDB/pull/2401) 152* Add nullable (and unordered, nullable) support to the smoke test. [#2405](https://github.com/TileDB-Inc/TileDB/pull/2405) 153* Initial sparse global order reader [#2395](https://github.com/TileDB-Inc/TileDB/pull/2395) 154* Remove sm.sub_partitioner_memory_budget [#2402](https://github.com/TileDB-Inc/TileDB/pull/2402) 155* Update the markdown documents for our new version of array schemas [#2416](https://github.com/TileDB-Inc/TileDB/pull/2416) 156* Sparse global order reader: no more result cell slab copy. [#2411](https://github.com/TileDB-Inc/TileDB/pull/2411) 157* Sparse global order reader: initial memory budget improvements. [#2413](https://github.com/TileDB-Inc/TileDB/pull/2413) 158* Optimization of result cell slabs generation for sparse global order reader. [#2414](https://github.com/TileDB-Inc/TileDB/pull/2414) 159* Remove selective unfiltering. [#2410](https://github.com/TileDB-Inc/TileDB/pull/2410) 160* Updated Azure Storage Lite SDK to 0.3.0 [#2419](https://github.com/TileDB-Inc/TileDB/pull/2419) 161* Respect memory budget for sparse global order reader. [#2425](https://github.com/TileDB-Inc/TileDB/pull/2425) 162* Use newer Azure patch for all platforms to solve missing header error [#2433](https://github.com/TileDB-Inc/TileDB/pull/2433) 163* increased diag output for differences reported by tiledb_unit (some of which may be reasonable) [#2437](https://github.com/TileDB-Inc/TileDB/pull/2437) 164* Adjustments to schema evolution new attribute reads [#2484](https://github.com/TileDB-Inc/TileDB/pull/2484) 165* Change `Quickstart` link in readthedocs/doxygen `index.rst` [#2448](https://github.com/TileDB-Inc/TileDB/pull/2448) 166* Initial sparse unordered with duplicates reader. [#2441](https://github.com/TileDB-Inc/TileDB/pull/2441) 167* Add calls to `malloc_trim` on context and query destruction linux to potentially reduce idle memory usage [#2443](https://github.com/TileDB-Inc/TileDB/pull/2443) 168* Add logger internals for `std::string` and `std::stringstream` for developer convenience [#2454](https://github.com/TileDB-Inc/TileDB/pull/2454) 169* Allow empty attribute writes. [#2461](https://github.com/TileDB-Inc/TileDB/pull/2461) 170* Refactored readers: serialization. [#2458](https://github.com/TileDB-Inc/TileDB/pull/2458) 171* Allow null data pointers for writes. [#2481](https://github.com/TileDB-Inc/TileDB/pull/2481) 172* Update backwards compatibility arrays for 2.3.0 [#2487](https://github.com/TileDB-Inc/TileDB/pull/2487) 173 174## Deprecations 175 176* Deprecate all `*_with_key` APIs. [#2245](https://github.com/TileDB-Inc/TileDB/pull/2245) [#2308](https://github.com/TileDB-Inc/TileDB/pull/2308) [#2412](https://github.com/TileDB-Inc/TileDB/pull/2412) 177 178## Bug fixes 179 180* Fix to correctly apply capnproto create_symlink avoidance patch [#2264](https://github.com/TileDB-Inc/TileDB/pull/2264) 181* The bug for calculating max_size_validity for var_size attribute caused incomplete query [#2266](https://github.com/TileDB-Inc/TileDB/pull/2266) 182* Always run ASAN with matching compiler versions [#2277](https://github.com/TileDB-Inc/TileDB/pull/2277) 183* Fix some loop bounds that reference non-existent elements [#2282](https://github.com/TileDB-Inc/TileDB/pull/2282) 184* Treating `std::vector` like an array; accessing an element that is not present to get its address. [#2276](https://github.com/TileDB-Inc/TileDB/pull/2276) 185* Fix buffer arguments in unit-curl.cc [#2287](https://github.com/TileDB-Inc/TileDB/pull/2287) 186* Stop loop iterations within limits of vector being initialized. [#2320](https://github.com/TileDB-Inc/TileDB/pull/2320) 187* Modify FindCurl_EP.cmake to work for WIN32 -EnableDebug builds [#2319](https://github.com/TileDB-Inc/TileDB/pull/2319) 188* Fixing test failure because of an uninitialized buffer. [#2386](https://github.com/TileDB-Inc/TileDB/pull/2386) 189* Change a condition that assumed MSVC was the only compiler for WIN32 [#2388](https://github.com/TileDB-Inc/TileDB/pull/2388) 190* Fix defects in buffer classes: read, set_offset, advance_offset [#2342](https://github.com/TileDB-Inc/TileDB/pull/2342) 191* Use CHECK_SAFE() to avoid multi-threaded conflict [#2394](https://github.com/TileDB-Inc/TileDB/pull/2394) 192* Use tiledb _SAFE() items when overlapping threads may invoke code [#2418](https://github.com/TileDB-Inc/TileDB/pull/2418) 193* Changes to address issues with default string dimension ranges in query [#2436](https://github.com/TileDB-Inc/TileDB/pull/2436) 194* Only set cmake policy CMP0076 if cmake version in use knows about it [#2463](https://github.com/TileDB-Inc/TileDB/pull/2463) 195* Fix handling curl REST request having all data in single call back [#2485](https://github.com/TileDB-Inc/TileDB/pull/2485) 196* Write queries should post start/end timestamps for REST arrays [#2492](https://github.com/TileDB-Inc/TileDB/pull/2492) 197 198## API additions 199 200* Introduce new `tiledb_experimental.h` c-api header for new feature that don't have a stabilized api yet [#2453](https://github.com/TileDB-Inc/TileDB/pull/2453) 201* Introduce new `tiledb_experimental` cpp-api header for new feature that don't have a stabilized api yet [#2453](https://github.com/TileDB-Inc/TileDB/pull/2462) 202 203### C API 204 205* Refactoring [get/set]_buffer APIs [#2315](https://github.com/TileDB-Inc/TileDB/pull/2315) 206* Add `tiledb_fragment_info_get_array_schema` functions for getting the array schema of a fragment [#2468](https://github.com/TileDB-Inc/TileDB/pull/2468) 207* Add `tiledb_schema_evolution_t` and new apis for schema evolution [#2426](https://github.com/TileDB-Inc/TileDB/pull/2426) 208 209### C++ API 210 211* Refactoring [get/set]_buffer APIs [#2399](https://github.com/TileDB-Inc/TileDB/pull/2399) 212* Add `FragmentInfo::array_schema` functions for getting the array schema of a fragment [#2468](https://github.com/TileDB-Inc/TileDB/pull/2468) 213* Add `ArraySchemaEvolution` to cpp_api and its unit tests are also added. [#2462](https://github.com/TileDB-Inc/TileDB/pull/2462) 214 215# TileDB v2.3.4 Release Notes 216 217## Improvements 218 219* `Query::set_layout`: setting the layout on the subarray. [#2451](https://github.com/TileDB-Inc/TileDB/pull/2451) 220* Allow empty attribute writes. [#2461](https://github.com/TileDB-Inc/TileDB/pull/2461) 221 222## Bug fixes 223 224* Fix deserialization of buffers in write queries with nullable var-length attributes [#2442](https://github.com/TileDB-Inc/TileDB/pull/2442) 225 226# TileDB v2.3.3 Release Notes 227 228## Improvements 229 230* Increase REST (TileDB Cloud) retry count from 3 to 25 to be inline with S3/GCS retry times [#2421](https://github.com/TileDB-Inc/TileDB/pull/2421) 231* Avoid unnecessary `est_result_size` computation in `must_split` [#2431](https://github.com/TileDB-Inc/TileDB/pull/2431) 232* Use newer Azure patch for all platforms to solve missing header error [#2433](https://github.com/TileDB-Inc/TileDB/pull/2433) 233 234## Bug fixes 235 236* Fix c-api error paths always resetting any alloced pointers to nullptr in-addition to deleting [#2427](https://github.com/TileDB-Inc/TileDB/pull/2427) 237 238# TileDB v2.3.2 Release Notes 239 240## Improvements 241* Support more env selectable options in both azure-windows.yml and azure-windows-release.yml [#2384](https://github.com/TileDB-Inc/TileDB/pull/2384) 242* Enable Azure/Serialization for windows CI artifacts [#2400](https://github.com/TileDB-Inc/TileDB/pull/2400) 243 244## Bug fixes 245* Correct check for last offset position so that undefined memory is not accessed. [#2390](https://github.com/TileDB-Inc/TileDB/pull/2390) 246* Fix ch8416, failure to read array written with tiledb 2.2 via REST [#2404](https://github.com/TileDB-Inc/TileDB/pull/2404) 247* Fix ch7582: use the correct buffer for validity deserialization [#2407](https://github.com/TileDB-Inc/TileDB/pull/2407) 248 249# TileDB v2.3.1 Release Notes 250 251## Improvements 252* Update bzip2 in windows build to 1.0.8 [#2332](https://github.com/TileDB-Inc/TileDB/pull/2332) 253* Fixing S3 build for OSX11 [#2339](https://github.com/TileDB-Inc/TileDB/pull/2339) 254* Fixing possible overflow in Dimension::tile_num [#2265](https://github.com/TileDB-Inc/TileDB/pull/2265) 255* Fixing tile extent calculations for signed integer domains [#2303](https://github.com/TileDB-Inc/TileDB/pull/2303) 256* Add support for cross compilation on OSX in superbuild [#2354](https://github.com/TileDB-Inc/TileDB/pull/2354) 257* Remove curl link args for cross compilation [#2359](https://github.com/TileDB-Inc/TileDB/pull/2359) 258* Enable MacOS arm64 release artifacts [#2360](https://github.com/TileDB-Inc/TileDB/pull/2360) 259* Add more stats for \`compute_result_coords\` path [#2366](https://github.com/TileDB-Inc/TileDB/pull/2366) 260* Support credentials refresh for AWS [#2376](https://github.com/TileDB-Inc/TileDB/pull/2376) 261 262## Bug fixes 263* Fixing intermittent metadata test failure [#2338](https://github.com/TileDB-Inc/TileDB/pull/2338) 264* Fix query condition validation check for nullable attributes with null conditions [#2344](https://github.com/TileDB-Inc/TileDB/pull/2344) 265* Multi-range single dimension query fix [#2347](https://github.com/TileDB-Inc/TileDB/pull/2347) 266* Rewrite `Dimension::overlap_ratio` [#2304](https://github.com/TileDB-Inc/TileDB/pull/2304) 267* Follow up fixes to floating point calculations for tile extents [#2341](https://github.com/TileDB-Inc/TileDB/pull/2341) 268* Fix for set_null_tile_extent_to_range [#2361](https://github.com/TileDB-Inc/TileDB/pull/2361) 269* Subarray partitioner, unordered should be unordered, even for Hilbert. [#2377](https://github.com/TileDB-Inc/TileDB/pull/2377) 270 271# TileDB v2.3.0 Release Notes 272 273## Disk Format 274* Format version incremented to 9. [#2108](https://github.com/TileDB-Inc/TileDB/pull/2108) 275 276## Breaking behavior 277* The setting of \`sm.read_range_oob\` now defaults to \`warn\`, allowing queries to run with bounded ranges that errored before. [#2176](https://github.com/TileDB-Inc/TileDB/pull/2176) 278* Removes TBB as an optional dependency [#2181](https://github.com/TileDB-Inc/TileDB/pull/2181) 279 280## New features 281* Support TILEDB_DATETIME_{SEC,MS,US,NS} in arrow_io_impl.h [#2228](https://github.com/TileDB-Inc/TileDB/pull/2228) 282* Adds support for filtering query results on attribute values [#2141](https://github.com/TileDB-Inc/TileDB/pull/2141) 283* Adding support for time datatype dimension and attribute [#2140](https://github.com/TileDB-Inc/TileDB/pull/2140) 284* Add support for serialization of config objects [#2164](https://github.com/TileDB-Inc/TileDB/pull/2164) 285* Add C and C++ examples to the `examples/` directory for the `tiledb_fragment_info_t` APIs. [#2160](https://github.com/TileDB-Inc/TileDB/pull/2160) 286* supporting serialization (using capnproto) build on windows [#2100](https://github.com/TileDB-Inc/TileDB/pull/2100) 287* Config option "vfs.s3.sse" for S3 server-side encryption support [#2130](https://github.com/TileDB-Inc/TileDB/pull/2130) 288* Name attribute/dimension files by index. This is fragment-specific and updates the format version to version 9. [#2107](https://github.com/TileDB-Inc/TileDB/pull/2107) 289* Smoke Test, remove nullable structs from global namespace. [#2078](https://github.com/TileDB-Inc/TileDB/pull/2078) 290 291## Improvements 292* replace ReadFromOffset with ReadRange in GCS::read() to avoid excess gcs egress traffic [#2307](https://github.com/TileDB-Inc/TileDB/pull/2307) 293* Hilbert partitioning fixes [#2269](https://github.com/TileDB-Inc/TileDB/pull/2269) 294* Stats refactor [#2267](https://github.com/TileDB-Inc/TileDB/pull/2267) 295* Improve Cap'n Proto cmake setup for system installations [#2263](https://github.com/TileDB-Inc/TileDB/pull/2263) 296* Runtime check for minimum validity buffer size [#2261](https://github.com/TileDB-Inc/TileDB/pull/2261) 297* Enable partial vacuuming when vacuuming with timestamps [#2251](https://github.com/TileDB-Inc/TileDB/pull/2251) 298* Consolidation: de-dupe FragmentInfo [#2250](https://github.com/TileDB-Inc/TileDB/pull/2250) 299* Consolidation: consider non empty domain before start timestamp [#2248](https://github.com/TileDB-Inc/TileDB/pull/2248) 300* Add size details to s3 read error [#2249](https://github.com/TileDB-Inc/TileDB/pull/2249) 301* Consolidation: do not re-open array for each fragment [#2243](https://github.com/TileDB-Inc/TileDB/pull/2243) 302* Support back compat writes [#2230](https://github.com/TileDB-Inc/TileDB/pull/2230) 303* Serialization support for query conditions [#2240](https://github.com/TileDB-Inc/TileDB/pull/2240) 304* Make SubarrayPartitioner's member functions to return Status after calling Subarray::get_range_num. [#2235](https://github.com/TileDB-Inc/TileDB/pull/2235) 305* Update bzip2 super build version to 1.0.8 to address [CVE-2019-12900](https://nvd.nist.gov/vuln/detail/CVE-2019-12900) in libbzip2 [#2233](https://github.com/TileDB-Inc/TileDB/pull/2233) 306* Timestamp start and end for vacuuming and consolidation [#2227](https://github.com/TileDB-Inc/TileDB/pull/2227) 307* Fix memory leaks reported on ASAN when running with leak-detection. [#2223](https://github.com/TileDB-Inc/TileDB/pull/2223) 308* Use relative paths in consolidated fragment metadata [#2215](https://github.com/TileDB-Inc/TileDB/pull/2215) 309* Optimize `Subarray::compute_relevant_fragments` [#2216](https://github.com/TileDB-Inc/TileDB/pull/2216) 310* AWS S3: improve is_dir [#2209](https://github.com/TileDB-Inc/TileDB/pull/2209) 311* Add nullable string to nullable attribute example [#2212](https://github.com/TileDB-Inc/TileDB/pull/2212) 312* AWS S3: adding option to skip Aws::InitAPI [#2204](https://github.com/TileDB-Inc/TileDB/pull/2204) 313* Added additional stats for subarrays and subarray partitioners [#2200](https://github.com/TileDB-Inc/TileDB/pull/2200) 314* Introduces config parameter "sm.skip_est_size_partitioning" [#2203](https://github.com/TileDB-Inc/TileDB/pull/2203) 315* Add config to query serialization. [#2177](https://github.com/TileDB-Inc/TileDB/pull/2177) 316* Consolidation support for nullable attributes [#2196](https://github.com/TileDB-Inc/TileDB/pull/2196) 317* Adjust unit tests to reduce memory leaks inside the tests. [#2179](https://github.com/TileDB-Inc/TileDB/pull/2179) 318* Reduces memory usage in multi-range range reads [#2165](https://github.com/TileDB-Inc/TileDB/pull/2165) 319* Add config option \`sm.read_range_oob\` to toggle bounding read ranges to domain or erroring [#2162](https://github.com/TileDB-Inc/TileDB/pull/2162) 320* Windows msys2 build artifacts are no longer uploaded [#2159](https://github.com/TileDB-Inc/TileDB/pull/2159) 321* Add internal log functions to log at different log levels [#2161](https://github.com/TileDB-Inc/TileDB/pull/2161) 322* Parallelize Writer::filter_tiles [#2156](https://github.com/TileDB-Inc/TileDB/pull/2156) 323* Added config option "vfs.gcs.request_timeout_ms" [#2148](https://github.com/TileDB-Inc/TileDB/pull/2148) 324* Improve fragment info loading by parallelizing fragment_size requests [#2143](https://github.com/TileDB-Inc/TileDB/pull/2143) 325* Allow open array stats to be printed without read query [#2131](https://github.com/TileDB-Inc/TileDB/pull/2131) 326* Cleanup the GHA CI scripts - put common code into external shell scripts. [#2124](https://github.com/TileDB-Inc/TileDB/pull/2124) 327* Reduced memory consumption in the read path for multi-range reads. [#2118](https://github.com/TileDB-Inc/TileDB/pull/2118) 328* The latest version of `dev` was leaving behind a `test/empty_string3/`. This ensures that the directory is removed when `make check` is run. [#2113](https://github.com/TileDB-Inc/TileDB/pull/2113) 329* Migrating AZP CI to GA [#2111](https://github.com/TileDB-Inc/TileDB/pull/2111) 330* Cache non_empty_domain for REST arrays like all other arrays [#2105](https://github.com/TileDB-Inc/TileDB/pull/2105) 331* Add additional stats printing to breakdown read state initialization timings [#2095](https://github.com/TileDB-Inc/TileDB/pull/2095) 332* Places the in-memory filesystem under unit test [#1961](https://github.com/TileDB-Inc/TileDB/pull/1961) 333* Adds a Github Action to automate the HISTORY.md [#2075](https://github.com/TileDB-Inc/TileDB/pull/2075) 334* Change printfs in C++ examples to cout, edit C print statements to fix format warnings [#2226](https://github.com/TileDB-Inc/TileDB/pull/2226) 335 336## Deprecations 337* The following APIs have been deprecated: `tiledb_array_open_at`, `tiledb_array_open_at_with_key`, `tiledb_array_reopen_at`. [#2142](https://github.com/TileDB-Inc/TileDB/pull/2142) 338 339## Bug fixes 340* Fix a segfault on `VFS::ls` for the in-memory filesystem [#2255](https://github.com/TileDB-Inc/TileDB/pull/2255) 341* Fix rare read corruption in S3 [#2253](https://github.com/TileDB-Inc/TileDB/pull/2253) 342* Update some union initializers to use strict syntax [#2242](https://github.com/TileDB-Inc/TileDB/pull/2242) 343* Fix race within `S3::init_client` [#2247](https://github.com/TileDB-Inc/TileDB/pull/2247) 344* Expand accepted windows URIs. [#2237](https://github.com/TileDB-Inc/TileDB/pull/2237) 345* Write fix for unordered writes on nullable, fixed attributes. [#2241](https://github.com/TileDB-Inc/TileDB/pull/2241) 346* Fix tile extent to be reported as domain extent for sparse arrays with Hilbert ordering [#2231](https://github.com/TileDB-Inc/TileDB/pull/2231) 347* Do not consider option sm.read_range_oob for set_subarray() on Write queries [#2211](https://github.com/TileDB-Inc/TileDB/pull/2211) 348* Change avoiding generation of multiple, concatenated, subarray flattened data. [#2190](https://github.com/TileDB-Inc/TileDB/pull/2190) 349* Change mutex from basic to recursive [#2180](https://github.com/TileDB-Inc/TileDB/pull/2180) 350* Fixes a memory leak in the S3 read path [#2189](https://github.com/TileDB-Inc/TileDB/pull/2189) 351* Fixes a potential memory leak in the filter pipeline [#2185](https://github.com/TileDB-Inc/TileDB/pull/2185) 352* Fixes misc memory leaks in the unit tests [#2183](https://github.com/TileDB-Inc/TileDB/pull/2183) 353* Fix memory leak of \`tiledb_config_t\` in error path of \`tiledb_config_alloc\`. [#2178](https://github.com/TileDB-Inc/TileDB/pull/2178) 354* Fix check for null pointer in query deserialization [#2163](https://github.com/TileDB-Inc/TileDB/pull/2163) 355* Fixes a potential crash when retrying incomplete reads [#2137](https://github.com/TileDB-Inc/TileDB/pull/2137) 356* Fixes a potential crash when opening an array with consolidated fragment metadata [#2135](https://github.com/TileDB-Inc/TileDB/pull/2135) 357* Corrected a bug where sparse cells may be incorrectly returned using string dimensions. [#2125](https://github.com/TileDB-Inc/TileDB/pull/2125) 358* Fix segfault in serialized queries when partition is unsplittable [#2120](https://github.com/TileDB-Inc/TileDB/pull/2120) 359* Always use original buffer size in serialized read queries serverside. [#2115](https://github.com/TileDB-Inc/TileDB/pull/2115) 360* Fix an edge-case where a read query may hang on array with string dimensions [#2089](https://github.com/TileDB-Inc/TileDB/pull/2089) 361 362## API additions 363 364### C API 365* Added tiledb_array_set_open_timestamp_start and tiledb_array_get_open_timestamp_start [#2285](https://github.com/TileDB-Inc/TileDB/pull/2285) 366* Added tiledb_array_set_open_timestamp_end and tiledb_array_get_open_timestamp_end [#2285](https://github.com/TileDB-Inc/TileDB/pull/2285) 367* Addition of `tiledb_array_set_config` to directly assign a config to an array. [#2142](https://github.com/TileDB-Inc/TileDB/pull/2142) 368* tiledb_query_get_array now returns a deep-copy [#2184](https://github.com/TileDB-Inc/TileDB/pull/2184) 369* Added \`tiledb_serialize_config\` and \`tiledb_deserialize_config\` [#2164](https://github.com/TileDB-Inc/TileDB/pull/2164) 370* Add new api, `tiledb_query_get_config` to get a query's config. [#2167](https://github.com/TileDB-Inc/TileDB/pull/2167) 371* Removes non-default parameter in "tiledb_config_unset". [#2099](https://github.com/TileDB-Inc/TileDB/pull/2099) 372 373### C++ API 374* Added Array::set_open_timestamp_start and Array::open_timestamp_start [#2285](https://github.com/TileDB-Inc/TileDB/pull/2285) 375* Added Array::set_open_timestamp_end and Array::open_timestamp_end [#2285](https://github.com/TileDB-Inc/TileDB/pull/2285) 376* add Query::result_buffer_elements_nullable support for dims [#2238](https://github.com/TileDB-Inc/TileDB/pull/2238) 377* Addition of `tiledb_array_set_config` to directly assign a config to an array. [#2142](https://github.com/TileDB-Inc/TileDB/pull/2142) 378* Add new api, `Query.config()` to get a query's config. [#2167](https://github.com/TileDB-Inc/TileDB/pull/2167) 379* Removes non-default parameter in "Config::unset". [#2099](https://github.com/TileDB-Inc/TileDB/pull/2099) 380* Add support for a string-typed, variable-sized, nullable attribute in the C++ API. [#2090](https://github.com/TileDB-Inc/TileDB/pull/2090) 381 382# TileDB v2.2.9 Release Notes 383 384## Bug fixes 385* Fix rare read corruption in S3 [#2254](https://github.com/TileDB-Inc/TileDB/pull/2254) 386* Write fix for unordered writes on nullable, fixed attributes [#2241](https://github.com/TileDB-Inc/TileDB/pull/2241) 387 388# TileDB v2.2.8 Release Notes 389 390## Disk Format 391 392## Breaking C API changes 393 394## Breaking behavior 395 396## New features 397* Support TILEDB_DATETIME_{SEC,MS,US,NS} in arrow_io_impl.h [#2229](https://github.com/TileDB-Inc/TileDB/pull/2229) 398* Add support for serialization of config objects [#2164](https://github.com/TileDB-Inc/TileDB/pull/2164) 399* Add support for serialization of query config [#2177](https://github.com/TileDB-Inc/TileDB/pull/2177) 400 401## Improvements 402* Optimize `Subarray::compute_relevant_fragments` [#2218](https://github.com/TileDB-Inc/TileDB/pull/2218) 403* Reduces memory usage in multi-range range reads [#2165](https://github.com/TileDB-Inc/TileDB/pull/2165) 404* Add config option `sm.read_range_oob` to toggle bounding read ranges to domain or erroring [#2162](https://github.com/TileDB-Inc/TileDB/pull/2162) 405* Updates bzip2 to v1.0.8 on Linux/OSX [#2233](https://github.com/TileDB-Inc/TileDB/pull/2233) 406 407## Deprecations 408 409## Bug fixes 410* Fixes a potential memory leak in the filter pipeline [#2185](https://github.com/TileDB-Inc/TileDB/pull/2185) 411* Fixes misc memory leaks in the unit tests [#2183](https://github.com/TileDB-Inc/TileDB/pull/2183) 412* Fix memory leak of `tiledb_config_t` in error path of `tiledb_config_alloc`. [#2178](https://github.com/TileDB-Inc/TileDB/pull/2178) 413 414## API additions 415 416### C API 417* tiledb_query_get_array now returns a deep-copy [#2188](https://github.com/TileDB-Inc/TileDB/pull/2188) 418* Add new api,`tiledb_query_get_config` to get a query's config. [#2167](https://github.com/TileDB-Inc/TileDB/pull/2167) 419* Added `tiledb_serialize_config` and `tiledb_deserialize_config` [#2164](https://github.com/TileDB-Inc/TileDB/pull/2164) 420 421### C++ API 422* Add new api, `Query.config()` to get a query's config. [#2167](https://github.com/TileDB-Inc/TileDB/pull/2167) 423 424# TileDB v2.2.7 Release Notes 425 426## Improvements 427 428* Added config option vfs.gcs.request_timeout_ms [#2148](https://github.com/TileDB-Inc/TileDB/pull/2148) 429* Improve fragment info loading by parallelizing fragment_size requests [#2143](https://github.com/TileDB-Inc/TileDB/pull/2143) 430* Apply 'var_offsets.extra_element' mode to string dimension offsets too [#2145](https://github.com/TileDB-Inc/TileDB/pull/2145) 431 432# TileDB v2.2.6 Release Notes 433 434## Bug fixes 435 436* Fixes a potential crash when retrying incomplete reads [#2137](https://github.com/TileDB-Inc/TileDB/pull/2137) 437 438# TileDB v2.2.5 Release Notes 439 440## New features 441 442* Config option vfs.s3.sse for S3 server-side encryption support [#2130](https://github.com/TileDB-Inc/TileDB/pull/2130) 443 444## Improvements 445 446* Reduced memory consumption in the read path for multi-range reads. [#2118](https://github.com/TileDB-Inc/TileDB/pull/2118) 447* Cache non_empty_domain for REST arrays like all other arrays [#2105](https://github.com/TileDB-Inc/TileDB/pull/2105) 448* Add additional timer statistics for openning array for reads [#2027](https://github.com/TileDB-Inc/TileDB/pull/2027) 449* Allow open array stats to be printed without read query [#2131](https://github.com/TileDB-Inc/TileDB/pull/2131) 450 451## Bug fixes 452* Fixes a potential crash when opening an array with consolidated fragment metadata [#2135](https://github.com/TileDB-Inc/TileDB/pull/2135) 453* Corrected a bug where sparse cells may be incorrectly returned using string dimensions. [#2125](https://github.com/TileDB-Inc/TileDB/pull/2125) 454* Always use original buffer size in serialized read queries serverside. [#2115](https://github.com/TileDB-Inc/TileDB/pull/2115) 455* Fix segfault in serialized queries when partition is unsplittable [#2120](https://github.com/TileDB-Inc/TileDB/pull/2120) 456 457# TileDB v2.2.4 Release Notes 458 459## Improvements 460 461* Add additional stats printing to breakdown read state initialization timings [#2095](https://github.com/TileDB-Inc/TileDB/pull/2095) 462* Improve GCS multipart locking [#2087](https://github.com/TileDB-Inc/TileDB/pull/2087) 463 464## Bug fixes 465* Fix an edge-case where a read query may hang on array with string dimensions [#2089](https://github.com/TileDB-Inc/TileDB/pull/2089) 466* Fix mutex locking bugs on Windows due to unlocking on different thread and missing task join [#2077](https://github.com/TileDB-Inc/TileDB/pull/2077) 467 468### C++ API 469* Add support for a string-typed, variable-sized, nullable attribute in the C++ API. [#2090](https://github.com/TileDB-Inc/TileDB/pull/2090) 470 471# TileDB v2.2.3 Release Notes 472 473## New features 474 475* Add support for retrying REST requests that fail with certain http status code such as 503 [#2060](https://github.com/TileDB-Inc/TileDB/pull/2060) 476 477## Improvements 478 479* Parallelize across attributes when closing a write [#2048](https://github.com/TileDB-Inc/TileDB/pull/2048) 480* Support for dimension/attribute names that contain commonly reserved filesystem characters [#2047](https://github.com/TileDB-Inc/TileDB/pull/2047) 481* Remove unnecessary `is_dir` in `FragmentMetadata::store`, this can increase performance for s3 writes [#2050](https://github.com/TileDB-Inc/TileDB/pull/2050) 482* Improve S3 multipart locking [#2055](https://github.com/TileDB-Inc/TileDB/pull/2055) 483* Parallize loading fragments and array schema [#2061](https://github.com/TileDB-Inc/TileDB/pull/2061) 484 485# TileDB v2.2.2 Release Notes 486 487## New features 488 489* REST client support for caching redirects [#1919](https://github.com/TileDB-Inc/TileDB/pull/1919) 490 491## Improvements 492 493* Add `rest.creation_access_credentials_name` configuration parameter [#2025](https://github.com/TileDB-Inc/TileDB/pull/2025) 494 495## Bug fixes 496 497* Fixed ArrowAdapter export of string arrays with 64-bit offsets [#2037](https://github.com/TileDB-Inc/TileDB/pull/2037) 498* Fixed ArrowAdapter export of TILEDB_CHAR arrays with 64-bit offsets [#2039](https://github.com/TileDB-Inc/TileDB/pull/2039) 499 500## API additions 501 502### C API 503* Add `tiledb_query_set_config` to apply a `tiledb_config_t` to query-level parameters [#2030](https://github.com/TileDB-Inc/TileDB/pull/2030) 504 505### C++ API 506* Added `Query::set_config` to apply a `tiledb::Config` to query-level parameters [#2030](https://github.com/TileDB-Inc/TileDB/pull/2030) 507 508# TileDB v2.2.1 Release Notes 509 510## Breaking behavior 511 512* The tile extent can now be set to null, in which case internally TileDB sets the extent to the dimension domain range. [#1880](https://github.com/TileDB-Inc/TileDB/pull/1880) 513* The C++ API `std::pair<uint64_t, uint64_t> Query::est_result_size_var` has been changed to 1) a return type of `std::array<uint64_t, 2>` and 2) returns the offsets as a size in bytes rather than elements. [#1946](https://github.com/TileDB-Inc/TileDB/pull/1946) 514 515## New features 516 517* Support for nullable attributes. [#1895](https://github.com/TileDB-Inc/TileDB/pull/1895) [#1938](https://github.com/TileDB-Inc/TileDB/pull/1938) [#1948](https://github.com/TileDB-Inc/TileDB/pull/1948) [#1945](https://github.com/TileDB-Inc/TileDB/pull/1945) 518* Support for Hilbert order sorting for sparse arrays. [#1880](https://github.com/TileDB-Inc/TileDB/pull/1880) 519* Support for AWS S3 "AssumeRole" temporary credentials [#1882](https://github.com/TileDB-Inc/TileDB/pull/1882) 520* Support for zero-copy import/export with the Apache Arrow adapter [#2001](https://github.com/TileDB-Inc/TileDB/pull/2001) 521* Experimental support for an in-memory backend used with bootstrap option "--enable-memfs" [#1873](https://github.com/TileDB-Inc/TileDB/pull/1873) 522* Support for element offsets when reading var-sized attributes. [#1897] (https://github.com/TileDB-Inc/TileDB/pull/1897) 523* Support for an extra offset indicating the size of the returned data when reading var-sized attributes. [#1932] (https://github.com/TileDB-Inc/TileDB/pull/1932) 524* Support for 32-bit offsets when reading var-sized attributes. [#1950] (https://github.com/TileDB-Inc/TileDB/pull/1950) 525 526## Improvements 527 528* Optimized string dimension performance. 529* Added functionality to get fragment information from an array. [#1900](https://github.com/TileDB-Inc/TileDB/pull/1900) 530* Prevented unnecessary sorting when (1) there is a single fragment and (i) either the query layout is global order, or (ii) the number of dimensions is 1, and (2) when there is a single range for which the result coordinates have already been sorted. [#1880](https://github.com/TileDB-Inc/TileDB/pull/1880) 531* Added extra stats for consolidation. [#1880](https://github.com/TileDB-Inc/TileDB/pull/1880) 532* Disabled checking if cells are written in global order when consolidating, as it was redundant (the cells are already being read in global order during consolidation). [#1880](https://github.com/TileDB-Inc/TileDB/pull/1880) 533* Optimize consolidated fragment metadata loading [#1975](https://github.com/TileDB-Inc/TileDB/pull/1975) 534 535## Bug fixes 536 537* Fix tiledb_dimension_alloc returning a non-null pointer after error [#1959]((https://github.com/TileDB-Inc/TileDB/pull/1859) 538* Fixed issue with string dimensions and non-set subarray (which implies spanning the whole domain). There was an assertion being triggered. Now it works properly. 539* Fixed bug when checking the dimension domain for infinity or NaN values. [#1880](https://github.com/TileDB-Inc/TileDB/pull/1880) 540* Fixed bug with string dimension partitioning. [#1880](https://github.com/TileDB-Inc/TileDB/pull/1880) 541 542## API additions 543 544### C API 545 546* Added functions for getting fragment information. [#1900](https://github.com/TileDB-Inc/TileDB/pull/1900) 547* Added APIs for getting and setting ranges of queries using a dimension name. [#1920](https://github.com/TileDB-Inc/TileDB/pull/1920) 548 549### C++ API 550 551* Added class `FragmentInfo` and functions for getting fragment information. [#1900](https://github.com/TileDB-Inc/TileDB/pull/1900) 552* Added function `Dimension::create` that allows not setting a space tile extent. [#1880](https://github.com/TileDB-Inc/TileDB/pull/1880) 553* Added APIs for getting and setting ranges of queries using a dimension name. [#1920](https://github.com/TileDB-Inc/TileDB/pull/1920) 554* Changed `std::pair<uint64_t, uint64_t> Query::est_result_size_var` to `std::array<uint64_t, 2> Query::est_result_size_var`. Additionally, the size estimate for the offsets have been changed from elements to bytes. [#1946](https://github.com/TileDB-Inc/TileDB/pull/1946) 555 556# TileDB v2.2.0 Release Notes 557 558* This release was skipped due to an erroneous `2.2.0` git tag. 559 560# TileDB v2.1.6 Release Notes 561 562## Bug fixes 563* Fix deadlock in `ThreadPool::wait_or_work` [#1994](https://github.com/TileDB-Inc/TileDB/pull/1994) 564* Fix "[TileDB::ChunkedBuffer] Error: Cannot init chunk buffers; Total size must be non-zero." in read path [#1992](https://github.com/TileDB-Inc/TileDB/pull/1992) 565 566# TileDB v2.1.5 Release Notes 567 568## Improvements 569 570* Optimize consolidated fragment metadata loading [#1975](https://github.com/TileDB-Inc/TileDB/pull/1975) 571* Optimize `Reader::load_tile_offsets` for loading only relevant fragments [#1976](https://github.com/TileDB-Inc/TileDB/pull/1976) [#1983](https://github.com/TileDB-Inc/TileDB/pull/1983) 572* Optimize locking in `FragmentMetadata::load_tile_offsets` and `FragmentMetadata::load_tile_var_offsets` [#1979](https://github.com/TileDB-Inc/TileDB/pull/1979) 573* Exit early in `Reader::copy_coordinates`/`Reader::copy_attribute_values` when no results [#1984](https://github.com/TileDB-Inc/TileDB/pull/1984) 574 575## Bug fixes 576 577* Fix segfault in optimized `compute_results_sparse<char>` [#1969](https://github.com/TileDB-Inc/TileDB/pull/1969) 578* Fix GCS "Error:: Read object failed"[#1966](https://github.com/TileDB-Inc/TileDB/pull/1966) 579* Fix segfault in `ResultTile::str_coords_intersects` [#1981](https://github.com/TileDB-Inc/TileDB/pull/1981) 580 581# TileDB v2.1.4 Release Notes 582 583## Improvements 584 585* Optimize `ResultTile::compute_results_sparse<char>` resulting in significant performance increases in certain cases with string dimensions [#1963](https://github.com/TileDB-Inc/TileDB/pull/1963) 586 587# TileDB v2.1.3 Release Notes 588 589## Improvements 590 591* Optimized string dimension performance. [#1922](https://github.com/TileDB-Inc/TileDB/pull/1922) 592 593## Bug fixes 594 595* Updated the AWS SDK to v1.8.84 to fix an uncaught exception when using S3 [#1899](https://github.com/TileDB-Inc/TileDB/pull/1899)[TileDB-Py #409](https://github.com/TileDB-Inc/TileDB-Py/issues/409) 596* Fixed bug where a read on a sparse array may return duplicate values. [#1905](https://github.com/TileDB-Inc/TileDB/pull/1905) 597* Fixed bug where an array could not be opened if created with an array schema from an older version [#1889](https://github.com/TileDB-Inc/TileDB/pull/1889) 598* Fix compilation of TileDB Tools [#1926](https://github.com/TileDB-Inc/TileDB/pull/1926) 599 600# TileDB v2.1.2 Release Notes 601 602## Bug fixes 603 604* Fix ArraySchema not write protecting fill values for only schema version 6 or newer [#1868](https://github.com/TileDB-Inc/TileDB/pull/1868) 605* Fix segfault that may occur in the VFS read-ahead cache [#1871](https://github.com/TileDB-Inc/TileDB/pull/1871) 606 607# TileDB v2.1.1 Release Notes 608 609## Bug fixes 610 611* The result size estimatation routines will no longer return non-zero sizes that can not contain a single value. [#1849](https://github.com/TileDB-Inc/TileDB/pull/1849) 612* Fix serialization of dense writes that use ranges [#1860](https://github.com/TileDB-Inc/TileDB/pull/1860) 613* Fixed a crash from an unhandled SIGPIPE signal that may raise when using S3 [#1856](https://github.com/TileDB-Inc/TileDB/pull/1856) 614 615# TileDB v2.1.0 Release Notes 616 617## Breaking behavior 618 619* Empty dense arrays now return cells with fill values. Also the result estimator is adjusted to work properly with this new behavior. 620 621## New features 622 623* Added configuration option "sm.compute_concurrency_level" [#1766](https://github.com/TileDB-Inc/TileDB/pull/1766) 624* Added configuration option "sm.io_concurrency_level" [#1766](https://github.com/TileDB-Inc/TileDB/pull/1766) 625* Added configuration option "sm.sub_partitioner_memory_budget" [#1729](https://github.com/TileDB-Inc/TileDB/pull/1729) 626* Added configuration option "vfs.read_ahead_size" [#1785](https://github.com/TileDB-Inc/TileDB/pull/1785) 627* Added configuration option "vfs.read_ahead_cache_size" [#1785](https://github.com/TileDB-Inc/TileDB/pull/1785) 628* Added support for getting/setting Apache Arrow buffers from a Query [#1816](https://github.com/TileDB-Inc/TileDB/pull/1816) 629 630## Improvements 631 632* Source built curl only need HTTP support [#1712](https://github.com/TileDB-Inc/TileDB/pull/1712) 633* AWS SDK version bumped to 1.8.6 [#1718](https://github.com/TileDB-Inc/TileDB/pull/1718) 634* Split posix permissions into files and folers permissions [#1719](https://github.com/TileDB-Inc/TileDB/pull/1719) 635* Support seeking for CURL to allow redirects for posting to REST [#1728](https://github.com/TileDB-Inc/TileDB/pull/1728) 636* Changed default setting for `vfs.s3.proxy_scheme` from `https` to `http` to match common usage needs [#1759](https://github.com/TileDB-Inc/TileDB/pull/1759) 637* Enabled parallelization with native system threads when TBB is disabled [#1760](https://github.com/TileDB-Inc/TileDB/pull/1760) 638* Subarray ranges will be automatically coalesced as they are added [#1755](https://github.com/TileDB-Inc/TileDB/pull/1755) 639* Update GCS SDK to v1.16.0 to fixes multiple bugs reported [#1768](https://github.com/TileDB-Inc/TileDB/pull/1768) 640* Read-ahead cache for cloud-storage backends [#1785](https://github.com/TileDB-Inc/TileDB/pull/1785) 641* Allow multiple empty values at the end of a variable-length write [#1805](https://github.com/TileDB-Inc/TileDB/pull/1805) 642* Build system will raise overridable error if important paths contain regex character [#1808](https://github.com/TileDB-Inc/TileDB/pull/1808) 643* Lazily create AWS ClientConfiguration to avoid slow context creations for non S3 usage after the AWS SDK version bump [#1821](https://github.com/TileDB-Inc/TileDB/pull/1821) 644* Moved `Status`, `ThreadPool`, and `Logger` classes from folder `tiledb/sm` to `tiledb/common` [#1843](https://github.com/TileDB-Inc/TileDB/pull/1843) 645 646## Deprecations 647 648* Deprecated config option "sm.num_async_threads" [#1766](https://github.com/TileDB-Inc/TileDB/pull/1766) 649* Deprecated config option "sm.num_reader_threads" [#1766](https://github.com/TileDB-Inc/TileDB/pull/1766) 650* Deprecated config option "sm.num_writer_threads" [#1766](https://github.com/TileDB-Inc/TileDB/pull/1766) 651* Deprecated config option "vfs.num_threads" [#1766](https://github.com/TileDB-Inc/TileDB/pull/1766) 652* Support for MacOS older than 10.13 is being dropped when using the AWS SDK. Prebuilt Binaries now target 10.13 [#1753](https://github.com/TileDB-Inc/TileDB/pull/1753) 653* Use of Intel's Thread Building Blocks (TBB) will be discontinued in the future. It is now disabled by default [#1762](https://github.com/TileDB-Inc/TileDB/pull/1762) 654* No longer building release artifacts with Intel's Thread Building Blocks (TBB) enabled [#1825](https://github.com/TileDB-Inc/TileDB/pull/1825) 655 656## Bug fixes 657 658* Fixed bug in setting a fill value for var-sized attributes. 659* Fixed a bug where the cpp headers would always produce compile-time warnings about using the deprecated c-api "tiledb_coords()" [#1765](https://github.com/TileDB-Inc/TileDB/pull/1765) 660* Only serialize the Array URI in the array schema client side. [#1806](https://github.com/TileDB-Inc/TileDB/pull/1806) 661* Fix C++ api `consolidate_metadata` function uses incorrect config [#1841](https://github.com/TileDB-Inc/TileDB/pull/1841) [#1844](https://github.com/TileDB-Inc/TileDB/pull/1844) 662 663## API additions 664 665### C API 666 667* Added functions `tiledb_attribute_{set,get}_fill_value` to get/set default fill values 668 669### C++ API 670 671* Added functions `Attribute::{set,get}_fill_value` to get/set default fill values 672 673# TileDB v2.0.8 Release Notes 674 675## Improvements 676 677* Lazy initialization for GCS backend [#1752](https://github.com/TileDB-Inc/TileDB/pull/1752) 678* Add additional release artifacts which include disabling TBB [#1753](https://github.com/TileDB-Inc/TileDB/pull/1753) 679 680## Bug fixes 681 682* Fix crash during GCS backend initialization due to upstream bug. [#1752](https://github.com/TileDB-Inc/TileDB/pull/1752) 683 684# TileDB v2.0.7 Release Notes 685 686## Improvements 687 688* Various performance optimizations in the read path. [#1689](https://github.com/TileDB-Inc/TileDB/pull/1689) [#1692](https://github.com/TileDB-Inc/TileDB/pull/1692) [#1693](https://github.com/TileDB-Inc/TileDB/pull/1693) [#1694](https://github.com/TileDB-Inc/TileDB/pull/1694) [#1695](https://github.com/TileDB-Inc/TileDB/pull/1695) 689* Google Cloud SDK bumped to 1.14. [#1687](https://github.com/TileDB-Inc/TileDB/pull/1687), [#1742](https://github.com/TileDB-Inc/TileDB/pull/1742) 690 691## Bug fixes 692 693* Fixed error "Error: Out of bounds read to internal chunk buffer of size 65536" that may occur when writing var-sized attributes. [#1732](https://github.com/TileDB-Inc/TileDB/pull/1732) 694* Fixed error "Error: Chunk read error; chunk unallocated error" that may occur when reading arrays with more than one dimension. [#1736](https://github.com/TileDB-Inc/TileDB/pull/1736) 695* Fix Catch2 detection of system install [#1733](https://github.com/TileDB-Inc/TileDB/pull/1733) 696* Use libtiledb-detected certificate path for Google Cloud Storage, for better linux binary/wheel portability. [#1741](https://github.com/TileDB-Inc/TileDB/pull/1741) 697* Fixed a small memory leak when opening arrays. [#1690](https://github.com/TileDB-Inc/TileDB/pull/1690) 698* Fixed an overflow in the partioning path that may result in a hang or poor read performance. [#1725](https://github.com/TileDB-Inc/TileDB/pull/1725)[#1707](https://github.com/TileDB-Inc/TileDB/pull/1707) 699* Fix compilation on gcc 10.1 for blosc [#1740](https://github.com/TileDB-Inc/TileDB/pull/1740) 700* Fixed a rare hang in the usage of `load_tile_var_sizes`. [#1748](https://github.com/TileDB-Inc/TileDB/pull/1748) 701 702 703# TileDB v2.0.6 Release Notes 704 705## Improvements 706 707* Add new config option `vfs.file.posix_permissions`. [#1710](https://github.com/TileDB-Inc/TileDB/pull/1710) 708 709## Bug fixes 710 711* Return possible env config variables in config iter [#1714](https://github.com/TileDB-Inc/TileDB/pull/1714) 712 713# TileDB v2.0.5 Release Notes 714 715## Improvements 716 717* Don't include curl's linking to libz, avoids build issue with double libz linkage [#1682](https://github.com/TileDB-Inc/TileDB/pull/1682) 718 719# TileDB v2.0.4 Release Notes 720 721## Improvements 722 723* Fix typo in GCS cmake file for superbuild [#1665](https://github.com/TileDB-Inc/TileDB/pull/1665) 724* Don't error on GCS client init failure [#1667](https://github.com/TileDB-Inc/TileDB/pull/1667) 725* Don't include curl's linking to ssl, avoids build issue on fresh macos 10.14/10.15 installs [#1671](https://github.com/TileDB-Inc/TileDB/pull/1671) 726* Handle ubuntu's cap'n proto package not providing cmake targets [#1659](https://github.com/TileDB-Inc/TileDB/pull/1659) 727 728## Bug fixes 729 730* The C++ Attribute::create API now correctly builds from an STL array [#1670](https://github.com/TileDB-Inc/TileDB/pull/1670) 731* Allow opening arrays with read-only permission on posix filesystems [#1676](https://github.com/TileDB-Inc/TileDB/pull/1676) 732* Fixed build issue caused by passing std::string to an Aws method [#1678](https://github.com/TileDB-Inc/TileDB/pull/1678) 733 734# TileDB v2.0.3 Release Notes 735 736## Improvements 737 738* Add robust retries for S3 SLOW_DOWN errors [#1651](https://github.com/TileDB-Inc/TileDB/pull/1651) 739* Improve GCS build process [#1655](https://github.com/TileDB-Inc/TileDB/pull/1655) 740* Add generation of pkg-config file [#1656](https://github.com/TileDB-Inc/TileDB/pull/1656) 741* S3 should use HEADObject for file size [#1657](https://github.com/TileDB-Inc/TileDB/pull/1657) 742* Improvements to stats [#1652](https://github.com/TileDB-Inc/TileDB/pull/1652) 743* Add artifacts to releases from CI [#1663](https://github.com/TileDB-Inc/TileDB/pull/1663) 744 745## Bug fixes 746 747* Remove to unneeded semicolons noticed by the -pedantic flag [#1653](https://github.com/TileDB-Inc/TileDB/pull/1653) 748* Fix cases were TILEDB_FORCE_ALL_DEPS picked up system builds [#1654](https://github.com/TileDB-Inc/TileDB/pull/1654) 749* Allow errors to be show in cmake super build [#1658](https://github.com/TileDB-Inc/TileDB/pull/1658) 750* Properly check vacuum files and limit fragment loading [#1661](https://github.com/TileDB-Inc/TileDB/pull/1661) 751* Fix edge case where consolidated but unvacuumed array can have coordinates report twice [#1662](https://github.com/TileDB-Inc/TileDB/pull/1662) 752 753## API additions 754 755* Add c-api tiledb_stats_raw_dump[_str] function for raw stats dump [#1660](https://github.com/TileDB-Inc/TileDB/pull/1660) 756* Add c++-api Stats::raw_dump function for raw stats dump [#1660](https://github.com/TileDB-Inc/TileDB/pull/1660) 757 758# TileDB v2.0.2 Release Notes 759 760## Bug fixes 761* Fix hang on open array v1.6 [#1645](https://github.com/TileDB-Inc/TileDB/pull/1645) 762 763## Improvements 764* Allow empty values for variable length attributes [#1646](https://github.com/TileDB-Inc/TileDB/pull/1646) 765 766 767# TileDB v2.0.1 Release Notes 768 769## Improvements 770 771* Remove deprecated max buffer size APIs from unit tests [#1625](https://github.com/TileDB-Inc/TileDB/pull/1625) 772* Remove deprecated max buffer API from examples [#1626](https://github.com/TileDB-Inc/TileDB/pull/1626) 773* Remove zipped coords from examples [#1632](https://github.com/TileDB-Inc/TileDB/pull/1632) 774* Allow AWSSDK_ROOT_DIR override [#1637](https://github.com/TileDB-Inc/TileDB/pull/1637) 775 776## Deprecations 777 778## Bug fixes 779 780* Allow setting zipped coords multiple times [#1627](https://github.com/TileDB-Inc/TileDB/pull/1627) 781* Fix overflow in check_tile_extent [#1635](https://github.com/TileDB-Inc/TileDB/pull/1635) 782* Fix C++ Dimension API `{tile_extent,domain}_to_str`. [#1638](https://github.com/TileDB-Inc/TileDB/pull/1638) 783* Remove xlock in FragmentMetadata::store [#1639](https://github.com/TileDB-Inc/TileDB/pull/1639) 784 785## API additions 786 787# TileDB v2.0.0 Release Notes 788 789## Disk Format 790 791* Removed file `__coords.tdb` that stored the zipped coordinates in sparse fragments 792* Now storing the coordinate tiles on each dimension in separate files 793* Changed fragment name format from `__t1_t2_uuid` to `__t1_t2_uuid_<format_version>`. That was necessary for backwards compatibility 794 795## Breaking C API changes 796 797* Changed `domain` input of `tiledb_dimension_get_domain` to `const void**` (from `void**`). 798* Changed `tile_extent` input of `tiledb_dimension_get_tile_extent` to `const void**` (from `void**`). 799* Anonymous attribute and dimensions (i.e., empty strings for attribute/dimension names) is no longer supported. This is because now the user can set separate dimension buffers to the query and, therefore, supporting anonymous attributes and dimensions creates ambiguity in the current API. 800 801## Breaking behavior 802 803* Now the TileDB consolidation process does not clean up the fragments or array metadata it consolidates. This is (i) to avoid exclusively locking at any point during consolidation, and (ii) to enable fine-grained time traveling even in the presence of consolidated fragments or array metadata. Instead, we added a special vacuuming API which explicitly cleans up consolidated fragments or array metadata (with appropriate configuration parameters). The vacuuming functions briefly exclusively lock the array. 804 805## New features 806 807* Added string dimension support (currently only `TILEDB_STRING_ASCII`). 808* The user can now set separate coordinate buffers to the query. Also any subset of the dimensions is supported. 809* The user can set separate filter lists per dimension, as well as the number of values per coordinate. 810 811## Improvements 812 813* Added support for AWS Security Token Service session tokens via configuration option `vfs.s3.session_token`. [#1472](https://github.com/TileDB-Inc/TileDB/pull/1472) 814* Added support for indicating zero-value metadata by returning `value_num` == 1 from the `_get_metadatata` and `Array::get_metadata` APIs [#1438](https://github.com/TileDB-Inc/TileDB/pull/1438) (this is a non-breaking change, as the documented return of `value == nullptr` to indicate missing keys does not change)` 815* User can set coordinate buffers separately for write queries. 816* Added option to enable duplicate coordinates for sparse arrays [#1504](https://github.com/TileDB-Inc/TileDB/pull/1504) 817* Added support for writing at a timestamp by allowing opening an array at a timestamp (previously disabled). 818* Added special files with the same name as a fragment directory and an added suffix ".ok", to indicate a committed fragment. This improved the performance of opening an array on object stores significantly, as it avoids an extra REST request per fragment. 819* Added functionality to consolidation, which allows consolidating the fragment metadata footers in a single file by toggling a new config parameter. This leads to a huge performance boost when opening an array, as it avoids fetching a separate footer per fragment from storage. 820* Various reader parallelizations that boosted read performance significantly. 821* Configuration parameters can now be read from environmental variables. `vfs.s3.session_token` -> `TILEDB_VFS_S3_SESSION_TOKEN`. The prefix of `TILEDB_` is configurable via `config.env_var_prefix`. [#1600](https://github.com/TileDB-Inc/TileDB/pull/1600) 822 823## Deprecations 824* The TileDB tiledb_array_consolidate_metadata and tiledb_array_consolidate_metadata_with_key C-API routines have been deprecated and will be [removed entirely](https://github.com/TileDB-Inc/TileDB/issues/1591) in a future release. The tiledb_array_consolidate and tiledb_array_consolidate_with_key routines achieve the same behavior when the "sm.consolidation.mode" parameter of the configuration argument is equivalent to "array_meta". 825* The TileDB Array::consolidate_metadata CPP-API routine has been deprecated and will be [removed entirely](https://github.com/TileDB-Inc/TileDB/issues/1591) in a future release. The Array::consolidate routine achieves the same behavior when the "sm.consolidation.mode" parameter of the configuration argument is equivalent to "array_meta". 826 827## Bug fixes 828 829* Fixed bug in dense consolidation when the array domain is not divisible by the tile extents. 830 831## API additions 832 833* Added C API function `tiledb_array_has_metadata_key` and C++ API function `Array::has_metadata_key` [#1439](https://github.com/TileDB-Inc/TileDB/pull/1439) 834* Added C API functions `tiledb_array_schema_{set,get}_allows_dups` and C++ API functions `Array::set_allows_dups` and `Array::allows_dups` 835* Added C API functions `tiledb_dimension_{set,get}_filter_list` and `tiledb_dimension_{set,get}_cell_val_num` 836* Added C API functions `tiledb_array_get_non_empty_domain_from_{index,name}` 837* Added C API function `tiledb_array_vacuum` 838* Added C API functions `tiledb_array_get_non_empty_domain_var_size_from_{index,name}` 839* Added C API functions `tiledb_array_get_non_empty_domain_var_from_{index,name}` 840* Added C API function `tiledb_array_add_range_var` 841* Added C API function `tiledb_array_get_range_var_size` 842* Added C API function `tiledb_array_get_range_var` 843* Added C++ API functions `Dimension::set_cell_val_num` and `Dimension::cell_val_num`. 844* Added C++ API functions `Dimension::set_filter_list` and `Dimension::filter_list`. 845* Added C++ API functions `Array::non_empty_domain(unsigned idx)` and `Array::non_empty_domain(const std::string& name)`. 846* Added C++ API functions `Domain::dimension(unsigned idx)` and `Domain::dimension(const std::string& name)`. 847* Added C++ API function `Array::load_schema(ctx, uri)` and `Array::load_schema(ctx, uri, key_type, key, key_len)`. 848* Added C++ API function `Array::vacuum`. 849* Added C++ API functions `Array::non_empty_domain_var` (from index and name). 850* Added C++ API function `add_range` with string inputs. 851* Added C++ API function `range` with string outputs. 852* Added C++ API functions `Array` and `Context` constructors which take a c_api object to wrap. [#1623](https://github.com/TileDB-Inc/TileDB/pull/1623) 853 854## API removals 855 856# TileDB v1.7.7 Release Notes 857 858## Bug fixes 859 860* Fix expanded domain consolidation [#1572](https://github.com/TileDB-Inc/TileDB/pull/1572) 861 862# TileDB v1.7.6 Release Notes 863 864## New features 865 866* Add MD5 and SHA256 checksum filters [#1515](https://github.com/TileDB-Inc/TileDB/pull/1515) 867 868## Improvements 869 870* Added support for AWS Security Token Service session tokens via configuration option `vfs.s3.session_token`. [#1472](https://github.com/TileDB-Inc/TileDB/pull/1472) 871 872## Deprecations 873 874## Bug fixes 875 876* Fix new SHA1 for intel TBB in superbuild due to change in repository name [#1551](https://github.com/TileDB-Inc/TileDB/pull/1551) 877 878## API additions 879 880## API removals 881 882# TileDB v1.7.5 Release Notes 883 884## New features 885 886## Improvements 887 888* Avoid useless serialization of Array Metadata on close [#1485](https://github.com/TileDB-Inc/TileDB/pull/1485) 889* Update CONTRIBUTING and Code of Conduct [#1487](https://github.com/TileDB-Inc/TileDB/pull/1487) 890 891## Deprecations 892 893## Bug fixes 894 895* Fix deadlock in writes of TileDB Cloud Arrays [#1486](https://github.com/TileDB-Inc/TileDB/pull/1486) 896 897## API additions 898 899## API removals 900 901 902# TileDB v1.7.4 Release Notes 903 904## New features 905 906## Improvements 907 908* REST requests now will use http compression if available [#1479](https://github.com/TileDB-Inc/TileDB/pull/1479) 909 910## Deprecations 911 912## Bug fixes 913 914## API additions 915 916## API removals 917 918# TileDB v1.7.3 Release Notes 919 920## New features 921 922## Improvements 923 924* Array metadata fetching is now lazy (fetch on use) to improve array open performance [#1466](https://github.com/TileDB-Inc/TileDB/pull/1466) 925* libtiledb on Linux will no longer re-export symbols from statically linked dependencies [#1461](https://github.com/TileDB-Inc/TileDB/pull/1461) 926 927## Deprecations 928 929## Bug fixes 930 931## API additions 932 933## API removals 934 935# TileDB v1.7.2 Release Notes 936 937TileDB 1.7.2 contains bug fixes and several internal optimizations. 938 939## New features 940 941## Improvements 942 943* Added support for getting/setting array metadata via REST. [#1449](https://github.com/TileDB-Inc/TileDB/pull/1449) 944 945## Deprecations 946 947## Bug fixes 948 949* Fixed several REST query and deserialization bugs. [#1433](https://github.com/TileDB-Inc/TileDB/pull/1433), [#1437](https://github.com/TileDB-Inc/TileDB/pull/1437), [#1440](https://github.com/TileDB-Inc/TileDB/pull/1440), [#1444](https://github.com/TileDB-Inc/TileDB/pull/1444) 950* Fixed bug in setting certificate path on Linux for the REST client. [#1452](https://github.com/TileDB-Inc/TileDB/pull/1452) 951 952## API additions 953 954## API removals 955 956# TileDB v1.7.1 Release Notes 957 958TileDB 1.7.1 contains build system and bug fixes, and one non-breaking API update. 959 960## New features 961 962## Improvements 963 964## Deprecations 965 966## Bug fixes 967 968* Fixed bug in dense consolidation when the array domain is not divisible by the tile extents. [#1442](https://github.com/TileDB-Inc/TileDB/pull/1442) 969 970## API additions 971 972* Added C API function `tiledb_array_has_metadata_key` and C++ API function `Array::has_metadata_key` [#1439](https://github.com/TileDB-Inc/TileDB/pull/1439) 973* Added support for indicating zero-value metadata by returning `value_num` == 1 from the `_get_metadatata` and `Array::get_metadata` APIs [#1438](https://github.com/TileDB-Inc/TileDB/pull/1438) (this is a non-breaking change, as the documented return of `value == nullptr` to indicate missing keys does not change)` 974 975## API removals 976 977# TileDB v1.7.0 Release Notes 978 979TileDB 1.7.0 contains the new feature of array metadata, and numerous bugfixes. 980 981## New features 982 983* Added array metadata. [#1377](https://github.com/TileDB-Inc/TileDB/pull/1377) 984 985## Improvements 986 987* Allow writes to older-versioned arrays. [#1417](https://github.com/TileDB-Inc/TileDB/pull/1417) 988* Added overseen optimization to check the fragment non-empty domain before loading the fragment R-Tree. [#1395](https://github.com/TileDB-Inc/TileDB/pull/1395) 989* Use `major.minor` for SOVERSION instead of full `major.minor.rev`. [#1398](https://github.com/TileDB-Inc/TileDB/pull/1398) 990 991## Bug fixes 992 993* Numerous query serialization bugfixes and performance improvements. 994* Numerous tweaks to build strategy for libcurl dependency. 995* Fix crash in StorageManager destructor when GlobalState init fails. [#1393](https://github.com/TileDB-Inc/TileDB/pull/1393) 996* Fix Windows destructor crash due to missing unlock (mutex/refcount). [#1400](https://github.com/TileDB-Inc/TileDB/pull/1400) 997* Normalize attribute names in multi-range size estimation C API. [#1408](https://github.com/TileDB-Inc/TileDB/pull/1408) 998 999## API additions 1000 1001* Added C API functions `tiledb_query_get_{fragment_num,fragment_uri,fragment_timestamp_range}`. [#1396](https://github.com/TileDB-Inc/TileDB/pull/1396) 1002* Added C++ API functions `Query::{fragment_num,fragment_uri,fragment_timestamp_range}`. [#1396](https://github.com/TileDB-Inc/TileDB/pull/1396) 1003* Added C API function `tiledb_ctx_set_tag` and C++ API `Context::set_tag()`. [#1406](https://github.com/TileDB-Inc/TileDB/pull/1406) 1004* Add config support for S3 ca_path, ca_file, and verify_ssl options. [#1376](https://github.com/TileDB-Inc/TileDB/pull/1376) 1005 1006## API removals 1007 1008* Removed key-value functionality, `tiledb_kv_*` functions from the C API and Map and MapSchema from the C++ API. [#1415](https://github.com/TileDB-Inc/TileDB/pull/1415) 1009 1010# TileDB v1.6.3 Release Notes 1011 1012## Additions 1013 1014* Added config param `vfs.s3.logging_level`. [#1236](https://github.com/TileDB-Inc/TileDB/pull/1236) 1015 1016## Bug fixes 1017 1018* Fixed FP slice point-query with small (eps) gap coordinates. [#1384](https://github.com/TileDB-Inc/TileDB/pull/1384) 1019* Fixed several unused variable warnings in unit tests. [#1385](https://github.com/TileDB-Inc/TileDB/pull/1385) 1020* Fixed missing include in `subarray.h`. [#1374](https://github.com/TileDB-Inc/TileDB/pull/1374) 1021* Fixed missing virtual destructor in C++ API `schema.h`. [#1391](https://github.com/TileDB-Inc/TileDB/pull/1391) 1022* Fixed C++ API build error with clang regarding deleted default constructors. [#1394](https://github.com/TileDB-Inc/TileDB/pull/1394) 1023 1024# TileDB v1.6.2 Release Notes 1025 1026## Bug fixes 1027 1028* Fix incorrect version number listed in `tiledb_version.h` header file and doc page. 1029* Fix issue with release notes from 1.6.0 release. [#1359](https://github.com/TileDB-Inc/TileDB/pull/1359) 1030 1031# TileDB v1.6.1 Release Notes 1032 1033## Bug fixes 1034 1035* Bug fix in incomplete query behavior. [#1358](https://github.com/TileDB-Inc/TileDB/pull/1358) 1036 1037# TileDB v1.6.0 Release Notes 1038 1039The 1.6.0 release adds the major new feature of non-continuous range slicing, as well as a number of stability and usability improvements. Support is also introduced for datetime dimensions and attributes. 1040 1041## New features 1042 1043* Added support for multi-range reads (non-continuous range slicing) for dense and sparse arrays. 1044* Added support for datetime domains and attributes. 1045 1046## Improvements 1047 1048* Removed fragment metadata caching. [#1197](https://github.com/TileDB-Inc/TileDB/pull/1197) 1049* Removed array schema caching. [#1197](https://github.com/TileDB-Inc/TileDB/pull/1197) 1050* The tile MBR in the in-memory fragment metadata are organized into an R-Tree, speeding up tile overlap operations during subarray reads. [#1197](https://github.com/TileDB-Inc/TileDB/pull/1197) 1051* Improved encryption key validation process when opening already open arrays. Fixes issue with indefinite growing of the URI to encryption key mapping in `StorageManager` (the mapping is no longer needed). [#1197](https://github.com/TileDB-Inc/TileDB/pull/1197) 1052* Improved dense write performance in some benchmarks. [#1229](https://github.com/TileDB-Inc/TileDB/pull/1229) 1053* Support for direct writes without using the S3 multi-part API. Allows writing to 1054 Google Cloud Storage S3 compatibility mode. [#1219](https://github.com/TileDB-Inc/TileDB/pull/1219) 1055* Removed 256-character length limit from URIs. [#1288](https://github.com/TileDB-Inc/TileDB/pull/1288) 1056* Dense reads and writes now always require a subarray to be set, to avoid confusion. [#1320](https://github.com/TileDB-Inc/TileDB/pull/1320) 1057* Added query and array schema serialization API. [#1262](https://github.com/TileDB-Inc/TileDB/pull/1262) 1058 1059## Deprecations 1060 1061* The TileDB KV API has been deprecated and will be [removed entirely](https://github.com/TileDB-Inc/TileDB/issues/1258) in a future release. The KV mechanism will be removed when full support for string-valued dimensions has been added. 1062 1063## Bug fixes 1064 1065* Bug fix with amplification factor in consolidation. [#1275](https://github.com/TileDB-Inc/TileDB/pull/1275) 1066* Fixed thread safety issue in opening arrays. [#1252](https://github.com/TileDB-Inc/TileDB/pull/1252) 1067* Fixed floating point exception when writing fixed-length attributes with a large cell value number. [#1289](https://github.com/TileDB-Inc/TileDB/pull/1289) 1068* Fixed off-by-one limitation with floating point dimension tile extents. [#1314](https://github.com/TileDB-Inc/TileDB/pull/1314) 1069 1070## API additions 1071 1072### C API 1073 1074* Added functions `tiledb_query_{get_est_result_size, get_est_result_size_var, add_range, get_range_num, get_range}`. 1075* Added function `tiledb_query_get_layout` 1076* Added datatype `tiledb_buffer_t` and functions `tiledb_buffer_{alloc,free,get_type,set_type,get_data,set_data}`. 1077* Added datatype `tiledb_buffer_list_t` and functions `tiledb_buffer_list_{alloc,free,get_num_buffers,get_total_size,get_buffer,flatten}`. 1078* Added string conversion functions `tiledb_*_to_str()` and `tiledb_*_from_str()` for all public enum types. 1079* Added config param `vfs.file.enable_filelocks` 1080* Added datatypes `TILEDB_DATETIME_*` 1081* Added function `tiledb_query_get_array` 1082 1083 1084### C++ API 1085 1086* Added functions `Query::{query_layout, add_range, range, range_num, array}`. 1087## Breaking changes 1088 1089### C API 1090 1091* Removed ability to set `null` tile extents on dimensions. All dimensions must now have an explicit tile extent. 1092 1093### C++ API 1094 1095* Removed cast operators of C++ API objects to their underlying C API objects. This helps prevent inadvertent memory issues such as double-frees. 1096* Removed ability to set `null` tile extents on dimensions. All dimensions must now have an explicit tile extent. 1097* Changed argument `config` in `Array::consolidate()` from a const-ref to a pointer. 1098* Removed default includes of `Map` and `MapSchema`. To use the deprecated KV API temporarily, include `<tiledb/map.h>` explicitly. 1099 1100# TileDB v1.5.1 Release Notes 1101 1102## Improvements 1103 1104* Better handling of `{C,CXX}FLAGS` during the build. [#1209](https://github.com/TileDB-Inc/TileDB/pull/1209) 1105* Update libcurl dependency to v7.64.1 for S3 builds. [#1240](https://github.com/TileDB-Inc/TileDB/pull/1240) 1106 1107## Bug fixes 1108 1109* S3 SDK build error fix. [#1201](https://github.com/TileDB-Inc/TileDB/pull/1201) 1110* Fixed thread safety issue with ZStd compressor. [#1208](https://github.com/TileDB-Inc/TileDB/pull/1208) 1111* Fixed crash in consolidation due to accessing invalid entry [#1213](https://github.com/TileDB-Inc/TileDB/pull/1213) 1112* Fixed memory leak in C++ KV API. [#1247](https://github.com/TileDB-Inc/TileDB/pull/1247) 1113* Fixed minor bug when writing in global order with empty buffers. [#1248](https://github.com/TileDB-Inc/TileDB/pull/1248) 1114 1115# TileDB v1.5.0 Release Notes 1116 1117The 1.5.0 release focuses on stability, performance, and usability improvements, as well a new consolidation algorithm. 1118 1119## New features 1120 1121* Added an advanced, tunable consolidation algorithm. [#1101](https://github.com/TileDB-Inc/TileDB/pull/1101) 1122 1123## Improvements 1124 1125* Small tiles are now batched for larger VFS read operations, improving read performance in some cases. [#1093](https://github.com/TileDB-Inc/TileDB/pull/1093) 1126* POSIX error messages are included in log messages. [#1076](https://github.com/TileDB-Inc/TileDB/pull/1076) 1127* Added `tiledb` command-line tool with several supported actions. [#1081](https://github.com/TileDB-Inc/TileDB/pull/1081) 1128* Added build flag to disable internal statistics. [#1111](https://github.com/TileDB-Inc/TileDB/pull/1111) 1129* Improved memory overhead slightly by lazily allocating memory before checking the tile cache. [#1141](https://github.com/TileDB-Inc/TileDB/pull/1141) 1130* Improved tile cache utilization by removing erroneous use of cache for metadata. [#1151](https://github.com/TileDB-Inc/TileDB/pull/1151) 1131* S3 multi-part uploads are aborted on error. [#1166](https://github.com/TileDB-Inc/TileDB/pull/1166) 1132 1133## Bug fixes 1134 1135* Bug fix when reading from a sparse array with real domain. Also added some checks on NaN and INF. [#1100](https://github.com/TileDB-Inc/TileDB/pull/1100) 1136* Fixed C++ API functions `group_by_cell` and `ungroup_var_buffer` to treat offsets in units of bytes. [#1047](https://github.com/TileDB-Inc/TileDB/pull/1047) 1137* Several HDFS test build errors fixed. [#1092](https://github.com/TileDB-Inc/TileDB/pull/1092) 1138* Fixed incorrect indexing in `parallel_for`. [#1105](https://github.com/TileDB-Inc/TileDB/pull/1105) 1139* Fixed incorrect filter statistics counters. [#1112](https://github.com/TileDB-Inc/TileDB/pull/1112) 1140* Preserve anonymous attributes in `ArraySchema` copy constructor. [#1144](https://github.com/TileDB-Inc/TileDB/pull/1144) 1141* Fix non-virtual destructors in C++ API. [#1153](https://github.com/TileDB-Inc/TileDB/pull/1153) 1142* Added zlib dependency to AWS SDK EP. [#1165](https://github.com/TileDB-Inc/TileDB/pull/1165) 1143* Fixed a hang in the 'S3::ls()'. [#1183](https://github.com/TileDB-Inc/TileDB/pull/1182) 1144* Many other small and miscellaneous bug fixes. 1145 1146## API additions 1147 1148### C API 1149 1150* Added function `tiledb_vfs_dir_size`. 1151* Added function `tiledb_vfs_ls`. 1152* Added config params `vfs.max_batch_read_size` and `vfs.max_batch_read_amplification`. 1153* Added functions `tiledb_{array,kv}_encryption_type`. 1154* Added functions `tiledb_stats_{dump,free}_str`. 1155* Added function `tiledb_{array,kv}_schema_has_attribute`. 1156* Added function `tiledb_domain_has_dimension`. 1157 1158### C++ API 1159 1160* `{Array,Map}::consolidate{_with_key}` now takes a `Config` as an optional argument. 1161* Added function `VFS::dir_size`. 1162* Added function `VFS::ls`. 1163* Added `{Array,Map}::encryption_type()`. 1164* Added `{ArraySchema,MapSchema}::has_attribute()` 1165* Added `Domain::has_dimension()` 1166* Added constructor overloads for `Array` and `Map` to take a `std::string` encryption key. 1167* Added overloads for `{Array,Map}::{open,create,consolidate}` to take a `std::string` encryption key. 1168* Added untyped overloads for `Query::set_buffer()`. 1169 1170## Breaking changes 1171 1172### C API 1173 1174* Deprecated `tiledb_compressor_t` APIs from v1.3.x have been removed, replaced by the `tiledb_filter_list` API. [#1128](https://github.com/TileDB-Inc/TileDB/pull/1128) 1175* `tiledb_{array,kv}_consolidate{_with_key}` now takes a `tiledb_config_t*` as argument. 1176 1177### C++ API 1178 1179* Deprecated `tiledb::Compressor` APIs from v1.3.x have been removed, replaced by the `FilterList` API. [#1128](https://github.com/TileDB-Inc/TileDB/pull/1128) 1180 1181# TileDB v1.4.2 Release Notes 1182 1183## Bug fixes 1184 1185* Fixed support for config parameter values `sm.num_reader_threads` and `sm.num_writer_threads. User-specified values had been ignored for these parameters. [#1096](https://github.com/TileDB-Inc/TileDB/pull/1096) 1186* Fixed GCC 7 linker errors. [#1091](https://github.com/TileDB-Inc/TileDB/pull/1091) 1187* Bug fix in the case of dense reads in the presence of both dense and sparse fragments. [#1079](https://github.com/TileDB-Inc/TileDB/pull/1074) 1188* Fixed double-delta decompression bug on reads for uncompressible chunks. [#1074](https://github.com/TileDB-Inc/TileDB/pull/1074) 1189* Fixed unnecessary linking of shared zlib when using TileDB superbuild. [#1125](https://github.com/TileDB-Inc/TileDB/pull/1125) 1190 1191## Improvements 1192 1193* Added lazy creation of S3 client instance on first request. [#1084](https://github.com/TileDB-Inc/TileDB/pull/1084) 1194* Added config params `vfs.s3.aws_access_key_id` and `vfs.s3.aws_secret_access_key` for configure s3 access at runtime. [#1036](https://github.com/TileDB-Inc/TileDB/pull/1036) 1195* Added missing check if coordinates obey the global order in global order sparse writes. [#1039](https://github.com/TileDB-Inc/TileDB/pull/1039) 1196 1197# TileDB v1.4.1 Release Notes 1198 1199## Bug fixes 1200 1201* Fixed bug in incomplete queries, which should always return partial results. An incomplete status with 0 returned results must always mean that the buffers cannot even fit a single cell value. [#1056](https://github.com/TileDB-Inc/TileDB/pull/1056) 1202* Fixed performance bug during global order write finalization. [#1065](https://github.com/TileDB-Inc/TileDB/pull/1065) 1203* Fixed error in linking against static TileDB on Windows. [#1058](https://github.com/TileDB-Inc/TileDB/pull/1058) 1204* Fixed build error when building without TBB. [#1051](https://github.com/TileDB-Inc/TileDB/pull/1051) 1205 1206## Improvements 1207 1208* Set LZ4, Zlib and Zstd compressors to build in release mode. [#1034](https://github.com/TileDB-Inc/TileDB/pull/1034) 1209* Changed coordinates to always be split before filtering. [#1054](https://github.com/TileDB-Inc/TileDB/pull/1054) 1210* Added type-safe filter option methods to C++ API. [#1062](https://github.com/TileDB-Inc/TileDB/pull/1062) 1211 1212# TileDB v1.4.0 Release Notes 1213 1214The 1.4.0 release brings two new major features, attribute filter lists and at-rest array encryption, along with bugfixes and performance improvements. 1215 1216**Note:** TileDB v1.4.0 changes the on-disk array format. Existing arrays should be re-written using TileDB v1.4.0 before use. Starting from v1.4.0 and on, backwards compatibility for reading old-versioned arrays will be fully supported. 1217 1218## New features 1219 1220* All array data can now be encrypted at rest using AES-256-GCM symmetric encryption. [#968](https://github.com/TileDB-Inc/TileDB/pull/968) 1221* Negative and real-valued domain types are now fully supported. [#885](https://github.com/TileDB-Inc/TileDB/pull/885) 1222* New filter API for transforming attribute data with an ordered list of filters. [#912](https://github.com/TileDB-Inc/TileDB/pull/912) 1223* Current filters include: previous compressors, bit width reduction, bitshuffle, byteshuffle, and positive-delta encoding. 1224 * The bitshuffle filter uses an implementation by [Kiyoshi Masui](https://github.com/kiyo-masui/bitshuffle). 1225 * The byteshuffle filter uses an implementation by [Francesc Alted](https://github.com/Blosc/c-blosc) (from the Blosc project). 1226* Arrays can now be opened at specific timestamps. [#984](https://github.com/TileDB-Inc/TileDB/pull/984) 1227 1228## Deprecations 1229 1230* The C and C++ APIs for compression have been deprecated. The corresponding filter API should be used instead. The compression API will be removed in a future TileDB version. [#1008](https://github.com/TileDB-Inc/TileDB/pull/1008) 1231* Removed Blosc compressors (obviated by byteshuffle -> compressor filter list). 1232 1233## Bug fixes 1234 1235* Fix issue where performing a read query with empty result could cause future reads to return empty [#882](https://github.com/TileDB-Inc/TileDB/pull/882) 1236* Fix TBB initialization bug with multiple contexts [#898](https://github.com/TileDB-Inc/TileDB/pull/898) 1237* Fix bug in max buffer sizes estimation [#903](https://github.com/TileDB-Inc/TileDB/pull/903) 1238* Fix Buffer allocation size being incorrectly set on realloc [#911](https://github.com/TileDB-Inc/TileDB/pull/911) 1239 1240## Improvements 1241 1242* Added check if the coordinates fall out-of-bounds (i.e., outside the array domain) during sparse writes, and added config param `sm.check_coord_oob` to enable/disable the check (enabled by default). [#996](https://github.com/TileDB-Inc/TileDB/pull/996) 1243* Add config params `sm.num_reader_threads` and `sm.num_writer_threads` for separately controlling I/O parallelism from compression parallelism. 1244* Added contribution guidelines [#899](https://github.com/TileDB-Inc/TileDB/pull/899) 1245* Enable building TileDB in Cygwin environment on Windows [#890](https://github.com/TileDB-Inc/TileDB/pull/890) 1246* Added a simple benchmarking script and several benchmark programs [#889](https://github.com/TileDB-Inc/TileDB/pull/889) 1247* Changed C API and disk format integer types to have explicit bit widths. [#981](https://github.com/TileDB-Inc/TileDB/pull/981) 1248 1249## API additions 1250 1251### C API 1252 1253* Added `tiledb_{array,kv}_open_at`, `tiledb_{array,kv}_open_at_with_key` and `tiledb_{array,kv}_reopen_at`. 1254* Added `tiledb_{array,kv}_get_timestamp()`. 1255* Added `tiledb_kv_is_open` 1256* Added `tiledb_filter_t` `tiledb_filter_type_t`, `tiledb_filter_option_t`, and `tiledb_filter_list_t` types 1257* Added `tiledb_filter_*` and `tiledb_filter_list_*` functions. 1258* Added `tiledb_attribute_{set,get}_filter_list`, `tiledb_array_schema_{set,get}_coords_filter_list`, `tiledb_array_schema_{set,get}_offsets_filter_list` functions. 1259* Added `tiledb_query_get_buffer` and `tiledb_query_get_buffer_var`. 1260* Added `tiledb_array_get_uri` 1261* Added `tiledb_encryption_type_t` 1262* Added `tiledb_array_create_with_key`, `tiledb_array_open_with_key`, `tiledb_array_schema_load_with_key`, `tiledb_array_consolidate_with_key` 1263* Added `tiledb_kv_create_with_key`, `tiledb_kv_open_with_key`, `tiledb_kv_schema_load_with_key`, `tiledb_kv_consolidate_with_key` 1264 1265### C++ API 1266 1267* Added encryption overloads for `Array()`, `Array::open()`, `Array::create()`, `ArraySchema()`, `Map()`, `Map::open()`, `Map::create()` and `MapSchema()`. 1268* Added `Array::timestamp()` and `Array::reopen_at()` methods. 1269* Added `Filter` and `FilterList` classes 1270* Added `Attribute::filter_list()`, `Attribute::set_filter_list()`, `ArraySchema::coords_filter_list()`, `ArraySchema::set_coords_filter_list()`, `ArraySchema::offsets_filter_list()`, `ArraySchema::set_offsets_filter_list()` functions. 1271* Added overloads for `Array()`, `Array::open()`, `Map()`, `Map::open()` for handling timestamps. 1272 1273## Breaking changes 1274 1275### C API 1276 1277* Removed Blosc compressors. 1278* Removed `tiledb_kv_set_max_buffered_items`. 1279* Modified `tiledb_kv_open` to not take an attribute subselection, but instead take as input the 1280query type (similar to arrays). This makes the key-value store behave similarly to arrays, 1281which means that the key-value store does not support interleaved reads/writes any more 1282(an opened key-value store is used either for reads or writes, but not both). 1283* `tiledb_kv_close` does not flush the written items. Instead, `tiledb_kv_flush` must be 1284invoked explicitly. 1285 1286### C++ API 1287 1288* Removed Blosc compressors. 1289* Removed `Map::set_max_buffered_items`. 1290* Modified `Map::Map` and `Map::open` to not take an attribute subselection, but instead take as input the 1291query type (similar to arrays). This makes the key-value store behave similarly to arrays, 1292which means that the key-value store does not support interleaved reads/writes any more 1293(an opened key-value store is used either for reads or writes, but not both). 1294* `Map::close` does not flush the written items. Instead, `Map::flush` must be 1295invoked explicitly. 1296 1297# TileDB v1.3.2 Release Notes 1298 1299## Bug fixes 1300 1301* Fix read query bug from multiple fragments when query layout differs from array layout [#869](https://github.com/TileDB-Inc/TileDB/pull/869) 1302* Fix error when consolidating empty arrays [#861](https://github.com/TileDB-Inc/TileDB/pull/861) 1303* Fix bzip2 external project URL [#875](https://github.com/TileDB-Inc/TileDB/pull/875) 1304* Invalidate cached buffer sizes when query subarray changes [#882](https://github.com/TileDB-Inc/TileDB/pull/882) 1305 1306## Improvements 1307 1308* Add check to ensure tile extent greater than zero [#866](https://github.com/TileDB-Inc/TileDB/pull/866) 1309* Add `TILEDB_INSTALL_LIBDIR` CMake option [#858](https://github.com/TileDB-Inc/TileDB/pull/858) 1310* Remove `TILEDB_USE_STATIC_*` CMake variables from build [#871](https://github.com/TileDB-Inc/TileDB/pull/871) 1311* Allow HDFS init to succeed even if libhdfs is not found [#873](https://github.com/TileDB-Inc/TileDB/pull/873) 1312 1313# TileDB v1.3.1 Release Notes 1314 1315## Bug fixes 1316* Add missing checks when setting subarray for sparse writes [#843](https://github.com/TileDB-Inc/TileDB/pull/843) 1317* Fix dl linking build issue for C/C++ examples on Linux [#844](https://github.com/TileDB-Inc/TileDB/pull/844) 1318* Add missing type checks for C++ api Query object [#845](https://github.com/TileDB-Inc/TileDB/pull/845) 1319* Add missing check that coordinates are provided for sparse writes [#846](https://github.com/TileDB-Inc/TileDB/pull/846) 1320 1321## Improvements 1322 1323* Fixes to compile on llvm v3.5 [#831](https://github.com/TileDB-Inc/TileDB/pull/831) 1324* Add option disable building unittests [#836](https://github.com/TileDB-Inc/TileDB/pull/836) 1325 1326# TileDB v1.3.0 Release Notes 1327 1328Version 1.3.0 focused on performance, stability, documentation and API improvements/enhancements. 1329 1330## New features 1331 1332* New guided tutorial series added to documentation. 1333* Query times improved dramatically with internal parallelization using TBB (multiple PRs) 1334* Optional deduplication pass on writes can be enabled [#636](https://github.com/TileDB-Inc/TileDB/pull/636) 1335* New internal statistics reporting system to aid in performance optimization [#736](https://github.com/TileDB-Inc/TileDB/pull/736) 1336* Added string type support: ASCII, UTF-8, UTF-16, UTF-32, UCS-2, UCS-4 [#415](https://github.com/TileDB-Inc/TileDB/pull/415) 1337* Added `TILEDB_ANY` datatype [#446](https://github.com/TileDB-Inc/TileDB/pull/446) 1338* Added parallelized VFS read operations, enabled by default [#499](https://github.com/TileDB-Inc/TileDB/pull/499) 1339* SIGINT signals will cancel in-progress queries [#578](https://github.com/TileDB-Inc/TileDB/pull/578) 1340 1341## Improvements 1342 1343* Arrays must now be open and closed before issuing queries, which clarifies the TileDB consistency model. 1344* Improved handling of incomplete queries and variable-length attribute data. 1345* Added parallel S3, POSIX, and Win32 reads and writes, enabled by default. 1346* Query performance improvements with parallelism (using TBB as a dependency). 1347* Got rid of special S3 "directory objects." 1348* Refactored sparse reads, making them simpler and more amenable to parallelization. 1349* Refactored dense reads, making them simpler and more amenable to parallelization. 1350* Refactored dense ordered writes, making them simpler and more amenable to parallelization. 1351* Refactored unordered writes, making them simpler and more amenable to parallelization. 1352* Refactored global writes, making them simpler and more amenable to parallelization. 1353* Added ability to cancel pending background/async tasks. SIGINT signals now cancel pending tasks. 1354* Async queries now use a configurable number of background threads (default number of threads is 1). 1355* Added checks for duplicate coordinates and option for coordinate deduplication. 1356* Map usage via the C++ API `operator[]` is faster, similar to the `MapItem` path. 1357 1358## Bug Fixes 1359 1360* Fixed bugs with reads/writes of variable-sized attributes. 1361* Fixed file locking issue with simultaneous queries. 1362* Fixed S3 issues with simultaneous queries within the same context. 1363 1364## API additions 1365 1366### C API 1367 1368* Added `tiledb_array_alloc` 1369* Added `tiledb_array_{open, close, free}` 1370* Added `tiledb_array_reopen` 1371* Added `tiledb_array_is_open` 1372* Added `tiledb_array_get_query_type` 1373* Added `tiledb_array_get_schema` 1374* Added `tiledb_array_max_buffer_size` and `tiledb_array_max_buffer_size_var` 1375* Added `tiledb_query_finalize` function. 1376* Added `tiledb_ctx_cancel_tasks` function. 1377* Added `tiledb_query_set_buffer` and `tiledb_query_set_buffer_var` which sets a single attribute buffer 1378* Added `tiledb_query_get_type` 1379* Added `tiledb_query_has_results` 1380* Added `tiledb_vfs_get_config` function. 1381* Added `tiledb_stats_{enable,disable,reset,dump}` functions. 1382* Added `tiledb_kv_alloc` 1383* Added `tiledb_kv_reopen` 1384* Added `tiledb_kv_has_key` to check if a key exists in the key-value store. 1385* Added `tiledb_kv_free`. 1386* Added `tiledb_kv_iter_alloc` which takes as input a kv object 1387* Added `tiledb_kv_schema_{set,get}_capacity`. 1388* Added `tiledb_kv_is_dirty` 1389* Added `tiledb_kv_iter_reset` 1390* Added `sm.num_async_threads`, `sm.num_tbb_threads`, and `sm.enable_signal_handlers` config parameters. 1391* Added `sm.check_dedup_coords` and `sm.dedup_coords` config parameters. 1392* Added `vfs.num_threads` and `vfs.min_parallel_size` config parameters. 1393* Added `vfs.{s3,file}.max_parallel_ops` config parameters. 1394* Added `vfs.s3.multipart_part_size` config parameter. 1395* Added `vfs.s3.proxy_{scheme,host,port,username,password}` config parameters. 1396 1397### C++ API 1398 1399* Added `Array::{open, close}` 1400* Added `Array::reopen` 1401* Added `Array::is_open` 1402* Added `Array::query_type` 1403* Added `Context::cancel_tasks()` function. 1404* Added `Query::finalize()` function. 1405* Added `Query::query_type` 1406* Added `Query::has_results` 1407* Changed the return type of the `Query` setters to return the object reference. 1408* Added an extra `Query` constructor that omits the query type (this is inherited from the input array). 1409* Added `Map::{open, close}` 1410* Added `Map::reopen` 1411* Added `Map::is_dirty` 1412* Added `Map::has_key` to check for key presence. 1413* A `tiledb::Map` defined with only one attribute will allow implicit usage, e.x. `map[key] = val` instead of `map[key][attr] = val`. 1414* Added optional attributes argument in `Map::Map` and `Map::open` 1415* `MapIter` can be used to create iterators for a map. 1416* Added `MapIter::reset` 1417* Added `MapSchema::set_capacity` and `MapSchema::capacity` 1418* Support for trivially copyable objects, such as a custom data struct, was added. They will be backed by an `sizeof(T)` sized `char` attribute. 1419* `Attribute::create<T>` can now be used with compound `T`, such as `std::string` and `std::vector<T>`, and other 1420 objects such as a simple data struct. 1421* Added a `Dimension::create` factory function that does not take tile extent, 1422 which sets the tile extent to `NULL`. 1423* `tiledb::Attribute` can now be constructed with an enumerated type (e.x. `TILEDB_CHAR`). 1424* Added `Stats` class (wraps C API `tiledb_stats_*` functions) 1425* Added `Config::save_to_file` 1426 1427## Breaking changes 1428 1429### C API 1430 1431* `tiledb_query_finalize` must **always** be called before `tiledb_query_free` after global-order writes. 1432* Removed `tiledb_vfs_move` and added `tiledb_vfs_move_file` and `tiledb_vfs_move_dir` instead. 1433* Removed `force` argument from `tiledb_vfs_move_*` and `tiledb_object_move`. 1434* Removed `vfs.s3.file_buffer_size` config parameter. 1435* Removed `tiledb_query_get_attribute_status`. 1436* All `tiledb_*_free` functions now return `void` and do not take `ctx` as input (except for `tiledb_ctx_free`). 1437* Changed signature of `tiledb_kv_close` to take a `tiledb_kv_t*` argument instead of `tiledb_kv_t**`. 1438* Renamed `tiledb_domain_get_rank` to `tiledb_domain_get_ndim` to avoid confusion with matrix def of rank. 1439* Changed signature of `tiledb_array_get_non_empty_domain`. 1440* Removed `tiledb_array_compute_max_read_buffer_sizes`. 1441* Changed signature of `tiledb_{array,kv}_open`. 1442* Removed `tiledb_kv_iter_create` 1443* Renamed all C API functions that create TileDB objects from `tiledb_*_create` to `tiledb_*_alloc`. 1444* Removed `tiledb_query_set_buffers` 1445* Removed `tiledb_query_reset_buffers` 1446* Added query type argument to `tiledb_array_open` 1447* Changed argument order in `tiledb_config_iter_alloc`, `tiledb_ctx_alloc`, `tiledb_attribute_alloc`, `tiledb_dimension_alloc`, `tiledb_array_schema_alloc`, `tiledb_kv_schema_load`, `tiledb_kv_get_item`, `tiledb_vfs_alloc` 1448 1449### C++ API 1450 1451* Fixes with `Array::max_buffer_elements` and `Query::result_buffer_elements` to comply with the API docs. `pair.first` is the number of elements of the offsets buffer. If `pair.first` is 0, it is a fixed-sized attribute or coordinates. 1452* `std::array<T, N>` is backed by a `char` tiledb attribute since the size is not guaranteed. 1453* Headers have the `tiledb_cpp_api_` prefix removed. For example, the include is now `#include <tiledb/attribute.h>` 1454* Removed `VFS::move` and added `VFS::move_file` and `VFS::move_dir` instead. 1455* Removed `force` argument from `VFS::move_*` and `Object::move`. 1456* Removed `vfs.s3.file_buffer_size` config parameter. 1457* `Query::finalize` must **always** be called before going out of scope after global-order writes. 1458* Removed `Query::attribute_status`. 1459* The API was made header only to improve cross-platform compatibility. `config_iter.h`, `filebuf.h`, `map_item.h`, `map_iter.h`, and `map_proxy.h` are no longer available, but grouped into the headers of the objects they support. 1460* Previously a `tiledb::Map` could be created from a `std::map`, an anonymous attribute name was defined. This must now be explicitly defined: `tiledb::Map::create(tiledb::Context, std::string uri, std::map, std::string attr_name)` 1461* Removed `tiledb::Query::reset_buffers`. Any previous usages can safely be removed. 1462* `Map::begin` refers to the same iterator object. For multiple concurrent iterators, a `MapIter` should be manually constructed instead of using `Map::begin()` more than once. 1463* Renamed `Domain::rank` to `Domain::ndim` to avoid confusion with matrix def of rank. 1464* Added query type argument to `Array` constructor 1465* Removed iterator functionality from `Map`. 1466* Removed `Array::parition_subarray`. 1467 1468# TileDB v1.2.2 Release Notes 1469 1470## Bug fixes 1471 1472* Fix I/O bug on POSIX systems with large reads/writes ([#467](https://github.com/TileDB-Inc/TileDB/pull/467)) 1473* Memory overflow error handling (moved from constructors to init functions) ([#472](https://github.com/TileDB-Inc/TileDB/pull/472)) 1474* Memory leaks with realloc in case of error ([#472](https://github.com/TileDB-Inc/TileDB/pull/472)) 1475* Handle non-existent config param in C++ API ([#475](https://github.com/TileDB-Inc/TileDB/pull/475)) 1476* Read query overflow handling ([#485](https://github.com/TileDB-Inc/TileDB/pull/485)) 1477 1478## Improvements 1479 1480* Changed S3 default config so that AWS S3 just works ([#455](https://github.com/TileDB-Inc/TileDB/pull/455)) 1481* Minor S3 optimizations and error message fixes ([#462](https://github.com/TileDB-Inc/TileDB/pull/462)) 1482* Documentation additions including S3 usage ([#456](https://github.com/TileDB-Inc/TileDB/pull/456), [#458](https://github.com/TileDB-Inc/TileDB/pull/458), [#459](https://github.com/TileDB-Inc/TileDB/pull/459)) 1483* Various CI improvements ([#449](https://github.com/TileDB-Inc/TileDB/pull/449)) 1484 1485# TileDB v1.2.1 Release Notes 1486 1487## Bug fixes 1488 1489* Fixed TileDB header includes for all examples ([#409](https://github.com/TileDB-Inc/TileDB/pull/409)) 1490* Fixed TileDB library dynamic linking problem for C++ API ([#412](https://github.com/TileDB-Inc/TileDB/pull/412)) 1491* Fixed VS2015 build errors ([#424](https://github.com/TileDB-Inc/TileDB/pull/424)) 1492* Bug fix in the sparse case ([#434](https://github.com/TileDB-Inc/TileDB/pull/434)) 1493* Bug fix for 1D vector query layout ([#438](https://github.com/TileDB-Inc/TileDB/pull/438)) 1494 1495## Improvements 1496 1497* Added documentation to API and examples ([#410](https://github.com/TileDB-Inc/TileDB/pull/410), [#414](https://github.com/TileDB-Inc/TileDB/pull/414)) 1498* Migrated docs to Readthedocs ([#418](https://github.com/TileDB-Inc/TileDB/pull/418), [#420](https://github.com/TileDB-Inc/TileDB/pull/420), [#422](https://github.com/TileDB-Inc/TileDB/pull/422), [#423](https://github.com/TileDB-Inc/TileDB/pull/423), [#425](https://github.com/TileDB-Inc/TileDB/pull/425)) 1499* Added dimension domain/tile extent checks ([#429](https://github.com/TileDB-Inc/TileDB/pull/429)) 1500 1501 1502# TileDB v1.2.0 Release Notes 1503The 1.2.0 release of TileDB includes many new features, improvements in stability and performance, and two new language interfaces (Python and C++). There are also several breaking changes in the C API and on-disk format, documented below. 1504 1505**Important Note**: due to several improvements and changes in the underlying array storage mechanism, you will need to recreate any existing arrays in order to use them with TileDB v1.2.0. 1506 1507## New features 1508* Windows support. TileDB is now fully supported on Windows systems (64-bit Windows 7 and newer). 1509* Python API. We are very excited to announce the initial release of a Python API for TileDB. The Python API makes TileDB accessible to a much broader audience, allowing seamless integration with existing Python libraries such as NumPy, Pandas and the scientific Python ecosystem. 1510* C++ API. We've included a C++ API, which allows TileDB to integrate into modern C++ applications without having to write code towards the C API. The C++ API is more concise and provides additional compile time type safety. 1511* S3 object store support. You can now easily store, query, and manipulate your TileDB arrays on S3 API compatibile object stores, including Amazon's AWS S3 service. 1512* Virtual filesystem interface. The TileDB API now exposes a virtual filesystem (or VFS) interface, allowing you to perform tasks such as file creation, deletion, reads, and appends without worrying about whether your files are stored on S3, HDFS, a POSIX or Windows filesystem, etc. 1513* Key-value store. TileDB now provides a key-value (meta) data storage abstraction. Its implementation is built upon TileDB's sparse arrays and inherits all the properties of TileDB sparse arrays. 1514 1515## Improvements 1516* Homebrew formula added for easier installation on macOS. Homebrew is now the perferred method for installing TileDB and its dependencies on macOS. 1517* Docker images updated to include stable/unstable/dev options, and easy configuration of additional components (e.g. S3 support). 1518* Tile cache implemented, which will greatly speed up repeated queries on overlapping regions of the same array. 1519* Ability to pass runtime configuration arguments to TileDB/VFS backends. 1520* Unnamed (or "anonymous") dimensions are now supported; having a single anonymous attribute is also supported. 1521* Concurrency bugfixes for several compressors. 1522* Correctness issue fixed in double-delta compressor for some datatypes. 1523* Better build behavior on systems with older GCC or CMake versions. 1524* Several memory leaks and overruns fixed with help of sanitizers. 1525* Many improved error condition checks and messages for easier debugging. 1526* Many other small bugs and API inconsistencies fixed. 1527 1528## C API additions 1529* `tiledb_config_*`: Types and functions related to the new configuration object and functionality. 1530* `tiledb_config_iter_*`: Iteration functionality for retieving parameters/values from the new configuration object. 1531* `tiledb_ctx_get_config()`: Function to get a configuration from a context. 1532* `tiledb_filesystem_t`: Filesystem type enum. 1533* `tiledb_ctx_is_supported_fs()`: Function to check for support for a given filesystem backend. 1534* `tiledb_error_t`, `tiledb_error_message()` and `tiledb_error_free()`: Type and functions for TileDB error messages. 1535* `tiledb_ctx_get_last_error()`: Function to get last error from context. 1536* `tiledb_domain_get_rank()`: Function to retrieve number of dimensions in a domain. 1537* `tiledb_domain_get_dimension_from_index()` and `tiledb_domain_get_dimension_from_name()`: Replaces dimension iterators. 1538* `tiledb_dimension_{create,free,get_name,get_type,get_domain,get_tile_extent}()`: Functions related to creation and manipulation of `tiledb_dimension_t` objects. 1539* `tiledb_array_schema_set_coords_compressor()`: Function to set the coordinates compressor. 1540* `tiledb_array_schema_set_offsets_compressor()`: Function to set the offsets compressor. 1541* `tiledb_array_schema_get_attribute_{num,from_index,from_name}()`: Replaces attribute iterators. 1542* `tiledb_query_create()`: Replaced many arguments with new `tiledb_query_set_*()` setter functions. 1543* `tiledb_array_get_non_empty_domain()`: Function to retrieve the non-empty domain from an array. 1544* `tiledb_array_compute_max_read_buffer_sizes()`: Function to compute an upper bound on the buffer sizes required for a read query. 1545* `tiledb_object_ls()`: Function to visit the children of a path. 1546* `tiledb_uri_to_path()`: Function to convert a file:// URI to a platform-native path. 1547* `TILEDB_MAX_PATH` and `tiledb_max_path()`: The maximum length for tiledb resource paths. 1548* `tiledb_kv_*`: Types and functions related to the new key-value store functionality. 1549* `tiledb_vfs_*`: Types and functions related to the new virtual filesystem (VFS) functionality. 1550 1551## Breaking changes 1552 1553### C API 1554* Rename `tiledb_array_metadata_t` -> `tiledb_array_schema_t`, and associated `tiledb_array_metadata_*` functions to `tiledb_array_schema_*`. 1555* Remove `tiledb_attribute_iter_t`. 1556* Remove `tiledb_dimension_iter_t`. 1557* Rename `tiledb_delete()`, `tiledb_move()`, `tiledb_walk()` to `tiledb_object_{delete,move,walk}()`. 1558* `tiledb_ctx_create`: Config argument added. 1559* `tiledb_domain_create`: Datatype argument removed. 1560* `tiledb_domain_add_dimension`: Name, domain and tile extent arguments replaced with single `tiledb_dimension_t` argument. 1561* `tiledb_query_create()`: Replaced many arguments with new `tiledb_query_set_*()` setter functions. 1562* `tiledb_array_create()`: Added array URI argument. 1563* `tiledb_*_free()`: All free functions now take a pointer to the object pointer instead of simply object pointer. 1564* The include files are now installed into a `tiledb` folder. The correct path is now `#include <tiledb/tiledb.h>` (or `#include <tiledb/tiledb>` for the C++ API). 1565 1566### Resource Management 1567* Support for moving resources across previous VFS backends (local fs <-> HDFS) has been removed. A more generic implementation for this functionality with improved performance is planned for the next version of TileDB. 1568