Skip to content

Changelog

All notable changes to this project will be documented in this file, where appropriate the GitHub issue reference will be noted along with the change. Breaking changes will be clearly indicated with the icon.

The format is based on Keep a Changelog.

[0.26.1] - 2025-11-05

Removed

  • Python 3.10 regression tests, builds will continue but not recommended for use in production.
  • GET and SEARCH marked as deprecated to be removed in or after version 0.27.

Fixed

Changed

  • [#2804] [rugo] Replace parquet metadata reader (@joocer)
  • [#2806] Fix file descriptor leak and improve buffer handling in connectors and decoders (@Copilot)
  • [#2809] [sqlparser-rs] Version bump and planner changes (@joocer)
  • [#2814] Consolidate list_ops Cython files into single file with auto-generation (@joocer, @Copilot)
  • [#2816] Optimize initialization performance (@joocer, @Copilot)
  • [#2825] Improve interactions with external DBMSs (@joocer)
  • Numerous housekeeping changes to tidy the structure of the code and repository (@joocer,@Copilot)

Added

  • [#2771] Add @>> as ARRAY_CONTAINS_ALL operator (@joocer)
  • [#2769] Rewrite visibility filters to reduce complexity (@joocer)
  • [#2836] Add REPLACE and INITCAP functions (@joocer)
  • [#2832] Experimental support for free-threaded Python (@joocer)
  • [#2808] Support wildcards when querying local filesystem (@Copilot)
  • [#2807] Support protocol prefixes for paths (gs://, s3://) (@Copilot)
  • [#2851] Bespoke JSONL decoder with projection pushdown support (@joocer, @Copilot)
  • [#2885] Predicate compaction strategy (@joocer, @Copilot)
  • [#2892] Vendor RE2 (@joocer, @Copilot)

[0.25.0] - 2025-09-04

Removed

  • INT marked as deprecated to be removed in or after version 0.27.
  • Python 3.10 will be deprecated in version 0.26.

Fixed

Changed

Added

[0.24.1] - 2025-07-29

Fixed

[0.24.0] - 2025-07-28

Removed

  • Deprecated functions removed: LIST_CONTAINS, STR, STRING, FLOAT, TRY_NUMERIC, TRY_STRING, TRY_STRUCT, LEN

Fixed

Added

  • [#2651] Add current_timestamp function (@joocer)
  • [#2671] Support qualified wildcards mixed with explicit columns in SELECT (@joocer)
  • [#2679] Purge blob from caches on read error (@joocer)
  • [#2687] Cache parquet file statistics and use for read pruning (@joocer)
  • [#2692] Support predicate pushdowns when reading from S3 (@joocer)

Changed

[0.23.0] - 2025-06-29

Removed

Added

  • [#2631] Add support for DAY interval definitions to FOR clause (@joocer)

Fixed

  • [#2592] Cannot GROUP BY empty sets (@joocer)
  • [#2611] [CI] Refactor version incrementor as pre-commit (@joocer)
  • [#2614] Manually replace values to avoid multidimensional array confusion in numpy (@joocer)
  • [#2636] numpy object dtype definition incorrect in CROSS JOIN (@joocer)

Changed

  • [#2588] Pass through error messages from functions (@joocer)
  • [#2594] [Optimizer] Combine OR chains of ANY equals conditions (@joocer)
  • [#2481] [Optimizer] Rewrite STARTS_WITH and ENDS_WITH to LIKE statements (@joocer)
  • [#2604] Log the post-optimized plan to the statistics (@joocer)
  • [#2607] Unqualified SEMI and ANTI JOINs are interpreted as LEFT (@joocer)
  • [#2609] Filter joins (SEMI and ANTI) internals rewritten (@joocer)
  • [#2616] Refactor nested loop join to use pyarrow buffers (@joocer)
  • [#2619] Refactor hash join variation of INNER JOIN and internal bloom filter to use pyarrow buffers (@joocer)
  • [#2627] Arrow native CROSS JOIN UNNEST with pushed filters (@joocer)
  • [#2445] LEFT JOIN rewrite (@joocer)

[0.22.0] - 2025-05-20

Removed

  • Deprecated function aliases removed NUMERIC, LIST_CONTAINS_ANY, LIST_CONTAINS_ALL, use DOUBLE, ARRAY_CONTAINS_ANY, ARRAY_CONTAINS_ALL instead. STRUCT removed as no longer required.
  • LIST_CONTAINS, STR, STRING, FLOAT, TRY_NUMERIC, TRY_STRING, TRY_STRUCT, LEN marked as deprecated to be removed in or after version 0.24.

Fixed

  • [#2547] INSTR optimization incorrectly applied in some cases (@joocer)
  • [#2554] Explicit column ordering (@joocer)
  • [#2545] Decimal scale and precision ignored in CAST (@joocer)
  • [#2556] STRUCT ARRAYS not correctly converted to JSON ARRAYs (@joocer)
  • [#2563] ANY/ALL Ops fail with single row datasets (@joocer)
  • [#2565] SHOW COLUMNS doesn't show extended type information (@joocer)

Changed

  • [#2535] Avoid conversion to numpy arrays for binary comparisons (@joocer)

[0.21.0] - 2025-03-30

Removed

  • Deprecated function aliases removed CEILING, ABSOLUTE, MAXIMUM, MINIMUM, use shorter form CEIL, ABS, MAX, MIN instead.
  • INNER JOIN UNNEST removed from this release

Added

Fixed

Changed

[0.20.0] - 2025-02-13

Added

Fixed

  • [#1954] JSON Accessors rewritten to support literal = document->key form (@joocer)
  • [#2190] [Clickbench] Resolve failing queries (partial) (@joocer)
  • [#2167] [CI] Use freezegun to reduce flaky tests (@joocer)
  • [#2247] Non-Existent SQL tables returned incorrect error (@joocer)
  • [#2231] Visibility Filters don't accept array literals (@joocer)
  • [#2300] Visibility Filters don't restrict when no filter provided (@joocer)
  • [#2299] [Fuzzer] Unable to sort by DECIMAL columns which contain NULL values (@joocer)
  • [#2302] [Fuzzer] IS TRUE isn't handled consistently by different connectors (@joocer)
  • [#2340] [Fuzzer] != with NULL isn't handled correctly by all connectors (@joocer)
  • [#2343] [Fuzzer] IS TRUE isn't handled consistently by different connectors (@joocer)

Changed

[0.19.0] - 2025-01-02

Added

Fixed

  • [#2085] LEFT/RIGHT side in a LEFT JOIN occasionally swapped. (@joocer)
  • [#2082] Buffer Pool eviction puts reader into an invalid state (@joocer)
  • [#2091] Command Line interface fails with unhelpful errors (@joocer)
  • [#2113] Improve reliability of NULLIF function (@joocer)
  • [#2128] GCS limited to 1000 files (@joocer)
  • [#2134] Nested JSON in NDJSON/JSONL incorrectly normalized (@joocer)
  • [#2144] NULLIF doesn't warn VARCHAR and BLOB won't match (@joocer)
  • [#2151] Slow aggregations on calculations (@joocer)
  • [#2177] Unable to determine type of nested identifiers (@joocer)
  • [#2159] Constant-folding wasn't always null-aware (@joocer)
  • [#2180] Parentheses in ORDER BY and GROUP BY clauses weren't handled (@joocer)
  • [#2181] Complex boolean functions weren't always null-aware (@joocer)

Changed

  • [#2129] Rewritten LEFT ANTI JOIN operator (@joocer)
  • [#2149] ANTI and SEMI joins moved to their own operator (@joocer)
  • [#2161] Minor performance improvements to LEVENSHTIEN function (@joocer)
  • [#2161] Empty morsels are only pushed through the plan if all morsels are empty (@joocer)
  • [#2163] Prefer 64bit indexes in Cython (@joocer)
  • [#2142] Accept negative scaling factors on CEIL and FLOOR functions (@joocer)

[0.18.0] - 2024-10-15

Added

Fixed

Changed

  • [#1990] Flag LIST_CONTAINS_ANY and LIST_CONTAINS_ALL as Deprecated (@joocer)
  • [#2013] MemoryPool based on Python's multiprocessing buffer for storage (@joocer)
  • [#2021] Error raised when subqueries expose columns with duplicate names. (@joocer)
  • [#2066] Reduce materialization of ARRAY columns. (@joocer)

[0.17.0] - 2024-09-05

Added

Fixed

Changed

[0.16.0] - 2024-07-20

Added

Fixed

[0.15.9] - 2024-06-29

Added

  • [#1778] Specialized INNER JOIN operator for INTEGERS and VARCHARS (@joocer)
  • [#1768] Date calculation predicate rewriter (optimizer) (@joocer)

Fixed

  • [#1788] Errors reading files reported as SQL syntax errors. (@joocer)

[0.15.8] - 2024-06-29

Added

Fixed

  • [#1700] Mac versions not building and publishing to PyPI (@joocer)

[0.15.5] - 2024-06-18

Changed

  • [#1759] -> and ->> operators able to parse JSON strings and bytes (@joocer)

[0.15.4] - 2024-06-17

Added

  • [#1746] Support legacy Mabel LZMA compressed JSONL files (@joocer)
  • [#1748] S3/MinIO Connector supports async reads (@joocer)

Changed

Fixed

  • [#1756] Sort incorrectly returning zero records when first morsel is empty (@joocer)

[0.15.3] - 2024-06-06

Changed

  • [#1715] More optimizations AND, OR and XOR aware (@joocer)
  • [#1717] Memcached notified when resource read from local buffers (@joocer)

[0.15.1] - 2024-05-31

Added

  • [#1685] [ClickBench] Include BLOB as column types for predicate pushdowns (@joocer)
  • [#1697] [ClickBench] Optimizer removes redundant operators (@joocer)
  • [#1698] [ClickBench] Replace LIKE conditions with STARTS_WITH and ENDS_WITH functions (@joocer)
  • [#1581] ANY operator supports literal lists (@joocer)
  • [#1690] Support REGEXP_REPLACE function (@joocer)

[0.15.0] - 2024-05-26

Added

Changed

Fixed

[0.14.1] - 2024-04-13

Added

  • [#1571] Initial COSINE_SIMILARITY support (ad hoc). (@joocer)

Fixed

  • [#1573] Parquet COUNT(*) returns unfiltered row count. (@joocer)

[0.14.0] - 2024-04-07

Added

Changed

Fixed

  • [#1462] Unhandled exception on empty datasets. (@joocer)
  • [#1465] Clashing column names not aliased correctly. (@joocer)
  • [#1445] SHOW EXTENDED COLUMNS not working as expected - changes made to data profile format. (@joocer)
  • [#1473] INNER JOIN statistics not correct. (@joocer)
  • [#1474] Unhelpful error message when SELECT * is mixed with column references (@joocer)
  • [#1487] Filters are not applied to scans on specific conditions involving JOINs (@joocer)
  • [#1480] INTERVAL cannot be compared to durations (@joocer)
  • [#1485] Improved handling of COUNT(*) when pushing to parquet and SQL (@joocer)
  • [#1513] LIST_CONTAINS_ANY performance (@joocer)

[0.13.3] - 2024-02-13

Added

  • [#1427] Boolean simplification optimization strategy. (@joocer)

Fixed

Changes

  • [#1421] Improve CROSS JOIN UNNEST performance. (@joocer)
  • [#1410] Whilst working on JOIN improvements, improvements to DISTINCT made. (@joocer)
  • Refactored SQL Fuzzer regression test script. (@joocer)

[0.13.0] - 2024-02-03

Added

  • [#1404] Optimize single element literal IN conditions (@joocer)

Fixed

Changed

[0.12.2] - 2024-01-10

Added

Fixed

Changed

  • [#1366] Subscript refactor for performance (~3x faster). (@joocer)

[0.12.0] - 2024-01-02

Fixed

  • [#1080] Windows regression test failures. (@joocer)
  • Soundex incorrectly evaluated empty strings as '0000'. (@joocer)

Changed

Added

Removed

  • Python 3.8 is no longer supported. (@joocer)

[0.11.0] - 2023-06-16

Fixed

  • [#1069] Minor improvements identified during code review of code to generate numeric series. (@joocer)
  • [#1072] Minor improvements identified during code review of code to handle dates and intervals. (@joocer)
  • [#1026] Removed pin to version 0.11 of PyArrow dependabot
  • [#1077] Removed pin to version 0.7.1 of DuckDB dependabot

Changed

Added

  • [#1034] Schemas added for the internal sample datasets. (@joocer)
  • [#1038] Able to pass SqlAlchemy Engine to the SQL Connectors, allowing for more complex authentication scenarios. (@joocer)
  • [#1065] Support integer division operator DIV. (@joocer)

[0.10.0] - 2023-05-03

Warnings

  • .to_df() will be replaced with .pandas() in version 0.11.

Fixed

  • [#929] Improved error messages for malformed temporal clauses. (@joocer)
  • [#735] (correction) Cursor fetchone and fetchmany step over the record set. (@joocer)
  • [#994] LIMIT didn't prevent additional files from being read after limit was met. (@joocer)
  • [#996] Performance issues with LIMIT and serialization steps. (@joocer)
  • [#1008] JOIN on a literal fails when attempting to find good match. (@joocer)
  • [#1006] Errors handling filenames with multiple dots in the name. (@joocer)
  • [#1010] Predicates not pushed for ZSTD compressed files. (@joocer)
  • [#1007] Wildcards not interpreted correctly in some projection pushdowns (@joocer)
  • [#1015] Column comparisons not working as expected in predicate pushdowns (@joocer)

Changed

Added

[0.9.3] - 2023-03-04

Fixed

  • [#916] Profile error on morsel with all nulls in column (@joocer)
  • Correctness of LRU-K algorithm (@joocer)
  • [#917] Comparisons failed on very long and skinny tables (@joocer)

[0.9.2] - 2023-02-28

Fixed

  • [#909] Divide by Zero error handling empty pages (@joocer)
  • [#912] Literal expressioned which evaluate to a boolean were ignored (@joocer)

Changed

  • [#901] Generate Series no longer accepts single numbers or IP ranges, provide explicit start or use | to test IP address containment (@joocer)
  • [#848] Collection and SQL Connectors dynamically size reads to fill target morsel size (@joocer)

[0.9.1] - 2023-02-23

Fixed

[0.9.0] - 2023-02-19

Fixed

  • [#797] Name collisons with aliases cause issues in ORDER BY. (@joocer)
  • [#833] Unhelpful error when no statement is provided (@joocer)
  • [#870] Repeated columns in GROUP BY not processed (@joocer)
  • [#873] 2 x CodeQL security issues (@joocer)

Changed

  • [#799] Chunk large blob reads. (@joocer)
  • [#812] Abstract the tree structure that plans are built from. (@joocer)
  • [#808] Split Logical and Physical planning (partial). (@joocer)
  • [#825] Remove HyperLogLog from profiling. (@joocer)
  • [#750] More CLI improvements. (@joocer)
  • [#589] Moved conditional imports out of program initialization (@joocer)
  • [#836] Use PyArrow 11s exposure of underlying date values in profiler (@joocer)
  • [#853] CaskDB replaces RocksDB as default KV store (@joocer)
  • [#855] Caches have been renamed and separated from KV Stores to discourage incorrect use; The Memcache Cache is now imported using from opteryx.managers.cache import MemcachedCache (@joocer)
  • [#857] Removed PyYAML install (@joocer)
  • [#865] Replaced third-party DATE_TRUNC implementation with a first-party implementation (@joocer)
  • [#861] Replaced third-party bitarray library with a first-party implementation (@joocer)
  • [#871] Consistently name internal variables relating to chunks of data to 'morsel' (technically breaking, but no user impact expected) (@joocer)
  • [#880] Minor performance improvements (@joocer)

Added

  • [#801] New helper function opteryx.query(). (@joocer)
  • [#818] Save query plans to disk (partial). (@joocer)
  • [#163] Initial support for SQL databases as a data source. (@joocer)
  • [#844] Materialize results as a Polars dataframe. (@joocer)
  • [#869] Introduce a SQL Fuzzer. (@joocer)
  • [#877] Initial experimental implementation of internal KV database, HadroDB (@joocer)

[0.8.3] - 2023-01-10

Fixed

  • [#782] Support literal predicates in JOIN conditions. (@joocer)

Changed

Added

[0.8.2] - 2023-01-06

Fixed

  • [#757] Multiple bugs in config manager. (@joocer)
  • [#769] ARRAY_AGG couldn't be nested. (@joocer)
  • [#775] Connection function .arrow() materializes before applying limit. (@joocer)

Changed

Added

[0.8.1] - 2022-12-30

Fixed

[0.8.0] - 2022-12-27

Fixed

  • [#703] ORDER BY columns not in SELECT clause. (@joocer)
  • [#712] Aggregates on literals when combined with a GROUP BY clause. (@joocer)
  • [#710] SEARCH mishandles pages with empty values in first row. (@joocer)
  • [#711] DATE_TRUNC is case sensitive. (@joocer)

Changed

  • [#707] First try to estimate unique values using the Distogram for SHOW EXTENDED COLUMNS. (@joocer)
  • [#707] SHOW EXTENDED COLUMNS creates histograms of 20 bins. (@joocer)
  • [#707] Distogram (data profiler) significant performance improvements. (@joocer)
  • [#722] Allow temporal FOR after alias AS clauses. (@joocer)
  • [#743] 'Did you mean' prompt for columns better suggestions when casing is different. (@joocer)

Added

  • [#515] Implement various new functions. (@joocer)
  • [#19] Initial support for CTE expressions. (@joocer)
  • [#204] Initial support predicate pushdowns. (@joocer)
  • [#721] Improved temporal range error messages. (@joocer)

[0.7.0] - 2022-12-02

Fixed

  • [#653] LIKE and FOR clauses cannot coexist in SHOW queries. (@joocer)
  • [#669] COUNT(*) cannot be mixed with other aggregates. (@joocer)
  • [#518] SELECT * and GROUP BY can't be used together. (@joocer)
  • [#689] IS comparisons cannot be combined with other comparisons when optimization is off. (@joocer)

Changed

Added

[0.6.0] - 2022-11-08

Fixed

  • [#568] Unable to perform aggregates on literals. (@joocer)
  • [#592] Dates not always handled correctly. (@joocer)
  • [#600] Parameterization when used on query batches fails. (@joocer)
  • [#580] Empty result sets have no column information. (@joocer)
  • [#548] 'did you mean' message restored for dataset WITH hints. (@joocer)
  • [#640] COUNT(*) shortcut only used when in uppercase. (@joocer)
  • [#645] (correction) null values not handled correctly in comparisions. (@joocer)
  • Problem installing on M1 Mac. (@joocer)
  • Support AND, OR, and XOR in SELECT statement. (@joocer)
  • [#646] Temporal clauses in incorrect place were ignored (@joocer)

Changed

  • [#566] Change from using SQLite3 to DuckDB for SQL comparision tests in Wrenchy-Bench. (@joocer)
  • [#584] (clarity) enable_page_management configuration and parameter renamed enable_page_defragmentation with some minor refactoring of approach to defragmentation. (@joocer)
  • (alignment) TIMESTAMP casting no longer supports casting from a number. (@joocer)
  • [#588] Integrate sqloxide into Opteryx to reduce lag with sqlparser-rs updates. (@joocer)
  • [#619] Page defragmentation moved to an Operator and positioned by the Optimizer. (@joocer)
  • (correction) cursor 'fetch*' methods return Python tuple, rather than Python lists. (@joocer)

Added

  • [#533] Support LIKE on SHOW FUNCTIONS, see sqlparser-rs/#620. (@joocer)
  • [#570] Query Optimizer rule to reduce steps in expression evaluation by partial elimination of negatives. (@joocer)
  • [#129] Support FOR clauses for all datasets. (@joocer)
  • [#543] Support 'type string' notation for casting values. (@joocer)
  • [#596] Optimizer replaces ORDER BY and LIMIT plan steps with a single 'HeapSort' plan step. (@joocer)
  • [#515] NULLIF function. (@joocer)
  • [#581] New SQL Battery test that tests results, and initial set of tests. (@joocer)
  • [#577] Hierarchical buffer pool and configuration. (@joocer)

[0.5.0] - 2022-10-02

Fixed

  • [#528] .shape() and .count() not working as expected. (@joocer)
  • Numbers expressed in the form +n not parsed correctly. (@joocer)

Changed

  • (alignment) .as_arrow() renamed to .arrow() to align to DuckDB naming. (@joocer)
  • (consistency) SHOW COLUMNS returns the column name in the name column, previously column_name (@joocer)
  • (correction) cursor 'fetch*' methods returns tuples rather than dictionaries as defaults, this is correcting a bug in PEP249 compatibility. (@joocer)
  • [#517] (security) Placeholder changed from '%s' to '?'. (@joocer)
  • [#522] Implementation of LRU-K(2) for cache evictions. (@joocer)
  • [#537] Significant refactor of Query Planner. (@joocer)

Added

  • [#397] Time Travel with '$planets' dataset. (@joocer)
  • [#519] Introduce a size limit on .as_arrow(). (@joocer)
  • [#324] Support IN UNNEST(). (@joocer)
  • [#386] Support SET statements. (@joocer)
  • [#531] Support SHOW VARIABLES and SHOW PARAMETERS. (@joocer)
  • [#464] Support LEFT JOIN <relation> USING (@joocer)
  • [#402] INNER JOIN ON supports multiple conditions (@joocer)
  • [#551] Document stores (MongoDb + FireStore) return '_id' column holding string version of document ID. (@joocer)
  • [#532] Runtime parameters are able to be altered using the SET statement. (@joocer)
  • [#524] Query Optimizer - conjunctive predicate splitter. (@joocer)

[0.4.1] - 2022-09-12

Fixed

  • Fixed missing __init__ file. (@joocer)

[0.4.0] - 2022-09-12

Added

Changed

  • [#457] (correction) null values are removed before performing INNER JOIN USING. (@joocer)

Fixed

  • [#448] VERSION() failed and missing from regression suite. (@joocer)
  • [#404] COALESCE fails for NaN values. (@joocer)
  • [#453] PyArrow bug with long lists creating new columns. (@joocer)
  • [#444] Very low cardinality INNER JOINS exceed memory allocation. (@joocer)
  • [#459] Functions lose some detail on non-first page. (@joocer)
  • [#465] Pages aren't matched to schema for simple queries. (@joocer)
  • [#468] Parquet reader shows some fields as "item". (@joocer)
  • [#471] Column aliases not correctly applied when the relation has an alias. (@joocer)
  • [#489] Intermittent behaviour on hash JOIN algorithm. (@joocer)

[0.3.0] - 2022-08-28

Added

  • [#196] Partial implementation of projection pushdown (Parquet Only). (@joocer)
  • [#41] Enable the results of functions to be used as parameters for other functions. (@joocer)
  • [#42] Enable inline operations. (@joocer)
  • [#330] Support SIMILAR TO alias for RegEx match. (@joocer)
  • [#331] Support SAFE_CAST alias for TRY_CAST. (@joocer)
  • [#419] Various simple functions (SIGN, SQRT, TITLE, REVERSE). (@joocer)
  • [#364] Support SOUNDEX function. (@joocer)
  • [#401] Support SHA-based hash algorithm functions. (@joocer)

Changed

  • (alignment) Paths to storage adapters has been updated to reflect 'connector' terminology.
  • (sensible defaults) Default behaviour changed from Mabel partitioning to no partitioning.
  • (correction) - Use of aliases defined in the SELECT clause can no longer be used in WHERE and GROUP BY clauses - this is a correction to align to standard SQL behaviour.
  • (correction) - Use of 'None' as an alias for null is no longer supported - this is a correction to align to standard SQL behaviour.
  • [#326] Prefer pyarrow's 'promote' over manually handling missing fields. (@joocer)
  • [#39] Rewrite Aggregation Node to use Pyarrow group_by(). (@joocer)
  • [#338] Remove Evaluation Node. (@joocer)
  • [#58] Performance of ORDER BY RAND() improved. (@joocer)

Fixed

  • [#334] All lists should be cast to lists of strings. (@joocer)
  • [#382] INNER JOIN on UNNEST relation. (@joocer)
  • [#320] Can't execute functions on results of GROUP BY. (@joocer)
  • [#399] Strings in double quotes aren't parsed. (@joocer)

[0.2.0] - 2022-07-31

Added

Changed

  • [#35] Table scan planning done during query planning. (@joocer)
  • [#173] Data not found raises different errors under different scenarios. (@joocer)
  • Implementation of LEFT and RIGHT functions to reduce execution time. (@joocer)
  • [#258] Code release approach. (@joocer)
  • [#295] Removed redundant projection when SELECT *. (@joocer)
  • [#297] Filters on SHOW COLUMNS execute before profiling. (@joocer)

Fixed

  • [#252] Planner should gracefully convert byte strings to ascii strings. (@joocer)
  • [#184] Schema changes cause unexpected and unhelpful failures. (@joocer)
  • [#261] Read fails if buffer cache is unavailable. (@joocer)
  • [#277] Cache errors should be transparent. (@joocer)
  • [#285] DISTINCT on nulls throws error. (@joocer)
  • [#281] SELECT on empty aggregates reports missing columns. (@joocer)
  • [#312] Invalid dates in FOR clauses treated as TODAY. (@joocer)

[0.1.0] - 2022-07-02

Added

  • [#165] Support S3/MinIO data stores for blobs. (@joocer)
  • FAKE dataset constructor (part of #179). (@joocer)
  • [#177] Support SHOW FULL COLUMNS to read entire datasets rather than just the first blob. (@joocer)
  • [#194] Functions that are abbreviations, should have the full name as an alias. (@joocer)
  • [#201] generate_series() supports CIDR expansion. (@joocer)
  • [#175] Support WITH (NO_CACHE) hint to disable using cache. (@joocer)
  • [#203] When reporting that a column doesn't exist, it should suggest likely correct columns. (@joocer)
  • 'Not' Regular Expression match operator, !~ added to supported set of operators. (@joocer)
  • [#226] Implement DATE_TRUNC function. (@joocer)
  • [#230] Allow addressing fields as numbers. (@joocer)
  • [#234] Implement SEARCH function. (@joocer)
  • [#237] Implement COALESCE function. (@joocer)

Changed

  • Blob-based readers (disk & GCS) moved from 'local' and 'network' paths to a new 'blob' path. (@joocer)
  • Query Execution rewritten. (@joocer)
  • [#20] Split query planner and query plan into different modules. (@joocer)
  • [#164] Split dataset reader into specific types. (@joocer)
  • Expression evaluation short-cuts execution when executing evaluations against an array of null. (@joocer)
  • [#244] Improve performance of IN test against literal lists. (@joocer)

Fixed

  • [#172] LIKE on non string column gives confusing error (@joocer)
  • [#179] Aggregate Node creates new metadata for each chunk (@joocer)
  • [#183] NOT doesn't display in plan correctly (@joocer)
  • [#182] Unable to evaluate valid filters (@joocer)
  • [#178] SHOW COLUMNS returns type OTHER when it can probably work out the type (@joocer)
  • [#128] JOIN fails, using PyArrow .join() (@joocer)
  • [#189] Explicit JOIN algorithm exceeds memory (@joocer)
  • [#199] SHOW EXTENDED COLUMNS blows memory allocations on large tables (@joocer)
  • [#169] Selection nodes in EXPLAIN have nested parentheses. (@joocer)
  • [#220] LIKE clause fails for columns that contain nulls. (@joocer)
  • [#222] Column of NULL detects as VARCHAR. (@joocer)
  • [#225] UNNEST does not assign a type to the column when all of the values are NULL. (@joocer)

[0.0.2] - 2022-06-03

Added

  • [#72] Configuration is now read from opteryx.yaml rather than the environment. (@joocer)
  • [#139] Gather statistics on planning reading of segments. (@joocer)
  • [#151] Implement SELECT table.*. (@joocer)
  • [#137] GENERATE_SERIES function. (@joocer)

Fixed

  • [#106] ORDER BY on qualified fields fails (@joocer)
  • [#103] ORDER BY after JOIN errors (@joocer)
  • [#110] SubQueries AS statement ignored (@joocer)
  • [#112] SHOW COLUMNS doesn't work for non sample datasets (@joocer)
  • [#113] Sample data has "NaN" as a string, rather than the value NaN (@joocer)
  • [#111] CROSS JOIN UNNEST should return a NONE when the list is empty (or NONE) (@joocer)
  • [#119] 'NoneType' object is not iterable error on UNNEST (@joocer)
  • [#127] Reading from segments appears to only read the first segment (@joocer)
  • [#132] Multiprocessing regressed Caching functionality (@joocer)
  • [#140] Appears to have read both frames rather than the latest frame (@joocer)
  • [#144] Multiple JOINS in one query aren't recognized (@joocer)

[0.0.1] - 2022-05-09

Added

  • Additional statistics recording the time taken to scan partitions (@joocer)
  • Support for FULL JOIN and RIGHT JOIN (@joocer)

Changed

  • Use PyArrow implementation for INNER JOIN and LEFT JOIN (@joocer)

Fixed

  • [#99] Grouping by a list gives an unhelpful error message (@joocer)
  • [#100] Projection ignores field qualifications (@joocer)

[0.0.0]

  • Initial Version