13 posts tagged with "caching"

Spice v2.4.0-rc.1 is now available! 🔥

Spice v2.4.0-rc.1 is the first release candidate for v2.4.0. It adds performance improvements, S3 event-driven ingestion, SQL results-cache warmup, and adaptive HTTP rate controls. The release also upgrades to DataFusion v55, Ballista v55, Arrow v59, Vortex v0.86, Iceberg v0.11, and Turso v0.81.

Highlights in v2.4.0-rc.1 include:

• Performance & Query Engine: DataFusion v55, Vortex v0.86.1, Iceberg v0.11.0, and Ballista v55 update query planning, storage, and distributed execution.
• S3 Event-Driven Ingestion — ingest newly created objects through SQS notifications.
• Cache Warmup and Shared Fetches — warm the SQL results cache after the first refresh and coalesce concurrent source fetches.
• Adaptive HTTP Rate Controls — bound request waits and coordinate request limits across instances.
• ORC Support — query ORC files through listing connectors.
• Hugging Face Datasets: Query and accelerate files from the Hugging Face Hub.

What's New in v2.4.0-rc.1​

Performance & Query Engine​

This release upgrades Apache DataFusion to the v55.2.0 dependency line and Apache Arrow to v59.3.0. It also upgrades Vortex to v0.86.1, Apache Iceberg to the v0.11.0 fork, and Apache Ballista to v55.

Apache DataFusion v55​

The DataFusion v55 release adds the following improvements:

• Sort pushdown and TopK pruning: Parquet scans reevaluate each unread row group as the threshold for ORDER BY ... LIMIT tightens. They skip groups that cannot contribute to the result. TopK pruning also supports multiple sort columns.
• Join planning: The optimizer converts eligible inner joins to semi joins and removes redundant sides of outer joins. It also orders filter predicates by estimated cost.
• Aggregation and expressions: Multi-column GROUP BY uses column-oriented storage for all supported key types, such as fixed-size binary UUIDs. More string functions preserve dictionary encoding, and IN lists use specialized paths for small integer types.
• Parquet reads: Scans skip nested fields that the declared schema does not contain. They also skip page-index reads when a file has no page index.
• Spill handling: Sorts bound the number of streams in a merge. If memory is insufficient, they spill the largest stream again in smaller batches.
• SQL diagnostics and functions: EXPLAIN accepts PostgreSQL-style options and FORMAT pgjson. New array functions cover element-wise addition, subtraction, scaling, sums, and averages.

Spice carries these changes through its query plans and preserves statistics across plan wrappers. Cayenne keeps Vortex scans below 10 MiB unsplit to avoid repeated footer reads. See #14612.

Vortex v0.79.0 to v0.86.1​

Spice v2.3.2 used the Vortex v0.79.0 fork. This upgrade covers the full upstream range from v0.79.0 through v0.86.1, not only the v0.86 changes:

• Types and arithmetic: Vortex adds native Map arrays, Arrow map conversion, and operations and compression for maps. It also adds union arrays and decimal addition, subtraction, multiplication, and division.
• Row selection: Piecewise-sequence indices represent contiguous selections without an expanded index for every row. Specialized paths handle chunked arrays, fixed-size lists, variable-length lists, and binary values.
• Scans and expressions: Layout scans gain a physical plan and expression optimization. Filters can pass through scalar functions with multiple arguments. Filters on wide lists restrict child elements to the selected range, and constant masks can resolve from metadata.
• Compression: Binary arrays support FSST compression with variable-length offsets. OnPair becomes a stable encoding for reads and gains a storage-backed dictionary. Vortex can convert run-end arrays of lists and decimals.
• File metadata: Files can store custom metadata. Readers cache decoded type descriptions, and file-format editions define the supported encodings and types.
• Memory and execution: Builders append nested values in batches, preserve list views, and propagate buffer allocators. Expression rewrites retain unchanged nodes, and row functions support batch execution.
• Correctness and validation: NULL handling changes cover dictionary predicates, BETWEEN bounds, and empty arrays. Readers add validation for footer offsets, compression metadata, and array indices. Other corrections cover nested scalar hashes, decimal operations, and variable-length binary selections above 4 GiB.

See the complete Vortex v0.79.0 to v0.86.1 changelog for every upstream change. Changes to standalone Vortex bindings and GPU execution do not imply new Spice features.

Apache Iceberg v0.11.0​

Spice updates its Iceberg reader and catalog integrations from v0.10.1 to the v0.11.0 fork for DataFusion 55 and Arrow 59. The upstream v0.11 changes extend reads and catalog compatibility:

• Iceberg v3 reads: The reader applies deletion vectors from Puffin files and carries row identifiers and sequence numbers through scans.
• Metadata-only scans: Metadata-only projections do not require data-column reads. Manifest reads reuse partition types, and positional-delete processing buffers runs instead of allocating a key per row.
• REST catalogs: Clients negotiate server-advertised endpoints and use session-scoped OAuth2 authentication.

The upgrade also aligns Iceberg storage with OpenDAL v0.58. See #14771.

Apache Ballista v55​

Spice.ai Enterprise feature. See the Enterprise documentation.

The Ballista v55 upgrade adds virtual-core resource accounting for distributed tasks. A protocol-version handshake detects incompatible schedulers and executors. Cancellation identifies tasks by their task IDs, and task-state records use an append-only model.

Schedulers and executors must run the same version. Upgrade all cluster components together.

S3 Event-Driven Ingestion​

S3 listing datasets can use refresh_mode: changes with S3 event notifications delivered through SQS:

datasets:
  - from: s3://my-bucket/events/
    name: events
    params:
      file_format: parquet
      s3_region: us-east-1
      s3_auth: iam_role
      s3_changes_queue_url: ${ secrets:events_queue_url }
    acceleration:
      enabled: true
      engine: cayenne
      mode: file
      refresh_mode: changes

New object notifications append the object's rows. A periodic listing backfill covers missed or expired notifications. Each dataset needs its own queue and permissions to read and delete SQS messages, alongside its S3 read/list permissions.

This is object ingestion: removal notifications are ignored by default. Set s3_on_object_removed: rebuild to rebuild the entire prefix when an object is removed. An overwrite of an already applied object key is not ingested again; use new object keys for incoming data. See #14121.

SQL Results-Cache Warmup and Shared Fetches​

The SQL results cache can persist query plan shapes and replay them after a dataset's first full or append refresh:

runtime:
  caching:
    sql_results:
      enabled: true
      warmup: on_first_refresh

For example, run these queries against an accelerated orders dataset with warmup enabled:

SELECT id, status FROM orders WHERE id = 1;
SELECT id, status FROM orders WHERE id = 2;

Spice records one query shape because only the equality-filter value differs. After a restart and the dataset's first full or append refresh, warmup reruns that shape with distinct id values from the refreshed dataset, filling the cache before the dataset becomes ready. Keep the local .spice/data directory across restarts, or configure runtime.state.location, to retain recorded query shapes.

Warmup replays up to ten distinct recorded query shapes and tries up to 1,024 distinct filter-value combinations per shape, stopping when the cache is full. A dataset stays not ready until warmup completes. Later refreshes do not repeat warmup. The feature requires the default plan-based cache key; cache_key_type: sql is incompatible. See #14178.

Concurrent cache-miss fetches for the same request now share a source fetch. SQL results caching also includes changes for tables updated during query execution and for stale results served while revalidation runs. HTTP dataset caching supports RFC 5861 stale-if-error handling. See #14142, #14710, #14708, and #14134.

SQL, search, and embedding caches now use Spice's sharded cache backend. Existing engine: moka and engine: pingora values are accepted for configuration compatibility but no longer select a backend.

Adaptive HTTP Rate Controls​

HTTP rate controls now adapt admission to upstream failures while staying within configured request limits. rate_control_acquire_timeout bounds how long a request waits for capacity and defaults to the connector's client timeout. rate_control_failure_threshold and rate_control_window control the response to upstream failures.

In Spice.ai Enterprise, instances that share a runtime.state.location coordinate per-second and per-minute limits through shared state; OSS instances keep these limits in memory independently. Concurrency limits remain local to each instance. Components sharing an upstream origin must use matching rate-control settings. See HTTP rate-control documentation and #14143.

ORC Files and Object Metadata Queries​

Listing connectors support file_format: orc. Object-store listing also uses predicates on metadata columns, including _last_modified, to narrow eligible objects. Queries that select only partition or metadata columns can use those values without reading file contents. See #14075, #14265, #14303, and #14116.

Hugging Face Datasets​

The new Hugging Face data connector queries and accelerates datasets from the Hugging Face Hub. It supports Parquet, CSV, TSV, JSON, and ORC files. Public datasets need no credentials:

datasets:
  - from: hf://datasets/stanfordnlp/imdb/plain_text/
    name: imdb
    acceleration:
      enabled: true

The location format is hf://datasets/<owner>/<dataset>[@<revision>][/<path>]. A path can select a file, a folder, or a glob. A revision can name a branch, a tag, or a commit. Use @~parquet to select the Hub's automatic Parquet conversion.

Set hf_token for private or gated datasets. Set hf_endpoint for a Hub mirror or proxy. Each scan reads one commit. Refreshes follow the selected branch, but the dataset keeps its registered schema until reload. See #14877.

Automatic Primary-Key Handling​

Cayenne keeps one row per primary_key without requiring an on_conflict policy. When a dataset sets time_column, the row with the newest time wins; without it, the last arrival wins. This applies to full and append refreshes as well as writes. Existing explicit conflict policies remain accepted during the deprecation period.

datasets:
  - from: s3://my-bucket/orders/
    name: orders
    time_column: updated_at
    params:
      file_format: parquet
    acceleration:
      enabled: true
      engine: cayenne
      mode: file
      primary_key: id

For a read-write dataset whose writes should stay in its acceleration, set acceleration.write_mode: acceleration. Its source need not support writes. This mode cannot be combined with a dataset that refreshes by changes.

Cayenne secondary indexes also support dynamic join filters, and their write handling covers inserts, updates, deletes, and refreshes. See #14726, #14282, and #14593.

PostgreSQL and MySQL replication, and MongoDB change streams, rejected Cayenne datasets that omitted on_conflict. Their validation still required an explicit upsert policy. These sources now accept Cayenne datasets with primary_key alone. See #14880.

For file-mode Cayenne datasets, append refreshes failed with a configuration that combined primary_key, time_column, and retention_sql. The refresh selected a version-resolution path that did not support retention. The refresh now resolves each key's newest version before Cayenne applies retention. See #14878.

Cayenne maintained aggregates now share one compact index of per-key contributions across views. Rebuilds capture concurrent writes and apply them after the scan. The aggregate budget uses 10% of a bounded query pool without the former 512 MiB cap. An unbounded pool retains the 512 MiB budget. See #14762.

Read Datasets from Published Snapshots​

Spice.ai Enterprise feature. See the Enterprise documentation.

A dataset can now read published acceleration snapshots directly, without configuring the original source connector:

datasets:
  - from: s3://my-bucket/spice/snapshots/orders/
    name: orders
    params:
      file_format: snapshot
      s3_region: us-east-1

Spice reads the snapshot metadata to select the engine, restores the published data, and checks for newer snapshots. The dataset is read-only. Snapshot-mode readers can also use S3 notifications delivered through SQS, with periodic checks retained for missed notifications. Each reader process needs its own queue.

This release includes changes to snapshot retries, slow-connection bootstrap, publication metadata, and coordination between snapshot archiving and Cayenne maintenance. Cayenne datasets with a datalake tier cannot create acceleration snapshots. See snapshot documentation, #14529, and #14335.

Connector and Protocol Updates​

• Connector status: ADBC, Databricks Spark Connect and SQL Warehouse, FlightSQL, Glue, HTTP/HTTPS, Iceberg, Localpod, and MongoDB are now Stable data connectors.
• MCP: support for specification version 2026-07-28, alongside the earlier protocol era. See #14043.
• GitHub: nested GraphQL pagination and rate-limit pacing updates; the default concurrency limit is now four. See #14179 and #14431.
• GitHub nested pages: Scans could return incomplete reviews or comments because pagination accepted a short page as complete. The connector now rejects incomplete connections and repeated cursors. It retries a failed nested page without another fetch of the outer page. Datasets with the same token also share the REST quota. See #14862.
• Iceberg REST: Clients could read an empty dataset because the catalog synthesized metadata without snapshots. The catalog now returns the source metadata for Iceberg datasets that Spice reads unchanged. Other datasets return 400 BadRequestException. Clients need their own storage credentials. See #14588 and Breaking Changes.

Other Fixes​

The release includes fixes in the following areas; the linked PRs provide details of the changes:

• Cayenne queries and writes: NULL-aware NOT IN, maintained aggregates, dynamic filters, partition-filter forwarding, memory-mode DML and retention, and primary-key handling across CDC checkpoints. See #14429, #14761, #14370, #14047, and #14344.
• Startup and reloads: retry datasets with unavailable sources, serve existing accelerations during source outages, and invalidate cached plans and results on catalog replacement or dataset unload. See #14623, #14624, #13914, and #14365.
• Federation: local evaluation of casts and functions whose source semantics differ, plus filter pushdown changes for DynamoDB, Cosmos DB, and MongoDB. See #14484, #14601, and #14419.
• HTTP and GraphQL: response-status handling during refresh, retry-budget handling, non-JSON gateway responses, and URL redaction in HTTP errors. See #13538, #14313, #14781, and #14490.
• Search: deletion of obsolete Elasticsearch chunks, non-finite embedding handling, and source-scan coordination during full-text refresh. See #13960, #13902, and #14663.
• Models and tools: tool-call-only assistant turns, required tool choices, streaming tool-use completion, model-load diagnostics, and propagation of the API-key principal into MCP tool calls. See #14232, #14460, #14548, and #14828.
• CDC shutdown and reconnects: source-position recording before accelerations close, and MySQL shared-stream reconnect handling. See #14702 and #14751.
• SQL weekdays: date_part('dow') aligns with EXTRACT(dow), with Sunday represented as zero. See #14796.
• Cayenne schema statistics: Decimal bounds could retain an old scale after schema evolution because maintenance published statistics from the previous schema. Cayenne now rejects statistics from an obsolete schema and keeps row counts conservative. See #14856.
• Vector search: vector_search planning failed after the DataFusion 55 upgrade because a second optimization pass tried to reorder a join with a dynamic filter. The planner now preserves that join's input order. See #14857.

Default accelerator: Datasets and views that enable acceleration without engine now use Cayenne. Explicit engine settings keep their behavior. Storage still defaults to memory. Set mode: file for persistent acceleration. See Breaking Changes for migration guidance.

Dependency Updates​

Contributors​

• @bjchambers
• @ewgenius
• @Jeadie
• @krinart
• @lukekim
• @peasee
• @phillipleblanc
• @sgrebnov

Breaking Changes​

Cayenne is the default accelerator on supported platforms. A dataset or view that omits acceleration.engine switches from Arrow to Cayenne. To retain Arrow, set engine: arrow explicitly before upgrading. Windows keeps Arrow as its default. Persistent datasets should continue to name their engine and use mode: file.

on_conflict is deprecated and scheduled for removal in v3.0. Cayenne automatically keeps one row per primary key, choosing the newest time_column value when configured, or the last arrival otherwise. Existing explicit policies remain supported during the deprecation period. Review those policies before removing them, especially drop or policies that reject conflicting rows.

on_conflict no longer routes writes to the acceleration. For read-write datasets whose writes should stay in the acceleration, use:

acceleration:
  enabled: true
  engine: cayenne
  write_mode: acceleration

This setting cannot be used with refresh_mode: changes, including a connector's default change-stream mode. The default write_through and write_back modes require a writable source.

Cache engine selection is retired. engine: moka and engine: pingora remain accepted but are ignored. Remove the field and use caching_policy to select eviction behavior.

GitHub connector default concurrency is four. Review explicit concurrency settings if your deployment relied on the previous default.

HTTP rate-control waits are bounded by default. Requests waiting for rate-control capacity now time out after the connector's client timeout. Set rate_control_acquire_timeout to a suitable duration, or 0 to retain the previous unbounded wait behavior.

Cayenne acceleration snapshots are unavailable for datalake-tier datasets. Review snapshot settings on datasets using cayenne_datalake_location; this release disables snapshotting that configuration.

Iceberg REST no longer synthesizes metadata for unsupported datasets. GET /v1/namespaces/{namespace}/tables/{table} returns 400 BadRequestException for accelerated datasets, views, and other datasets that Spice does not read unchanged from Iceberg. If a client used this endpoint for schema discovery, use SQL DESCRIBE or information_schema.columns instead. Query these datasets through /v1/sql or Arrow Flight SQL. For eligible Iceberg datasets, clients read the source metadata and need their own storage access. See the Get a table API.

Cookbook Updates​

The Spice Cookbook provides recipes to help you get started with Spice.

Upgrading​

To upgrade to v2.4.0-rc.1 once the release artifacts are available, use one of the following methods:

CLI:

spice upgrade v2.4.0-rc.1

Docker:

Pull the spiceai/spiceai:2.4.0-rc.1 image:

docker pull spiceai/spiceai:2.4.0-rc.1

For available tags, see DockerHub.

Helm:

helm repo update
helm upgrade spiceai spiceai/spiceai --version 2.4.0-rc.1

AWS Marketplace:

Spice is available in the AWS Marketplace. Marketplace availability follows its published versions.

What's Changed​

Changelog​

• fix(runtime): discard cached logical plans when a hot reload replaces a catalog (fixes #13910) by @claudespice in #13914
• fix(acceleration): let a schema repair correct a checkpoint without resetting the freshness clock (fixes #13817) by @claudespice in #13894
• fix(search): filter a chunked Elasticsearch delete on a field that can match the key (fixes #13714) by @claudespice in #13926
• fix(search): classify a partially non-finite embedding as unindexable on every backend (fixes #13872) by @claudespice in #13902
• fix: stabilize GitHub tests and bound GraphQL registration (fixes #13762) by @lukekim in #13939
• fix(postgres): decode versioned JSONB binary replication values by @phillipleblanc in #13962
• docs: require a reviewed Enhancement before any user-facing surface changes by @lukekim in #13970
• ci: upgrade spiceio setup action to v0.9.0 by @lukekim in #13971
• fix(postgres): preserve microseconds in timestamp writeback by @phillipleblanc in #13963
• feat(hash-index): verify the bloom filter's block index with Verus by @lukekim in #13777
• build(deps-dev): bump js-yaml by @dependabot in #13989
• docs: release notes for v2.3.0 by @bjchambers in #13999
• fix(ci): drop the dangling substrait-compliance submodule pointer by @bjchambers in #14002
• fix(cayenne): release the keyset bytes an abandoned PK checkout accounted (fixes #13668) by @grokspice in #13925
• fix(caching): keep a declared key from disabling eviction and stranding stale rows (fixes #13976) by @bjchambers in #13992
• Add Substrait compliance harness (IBM TPC-H Mode A + FlightSQL Mode B stub) by @lukekim in #13879
• ci: skip DynamoDB TPC-H benches in OSS testoperator dispatch by @phillipleblanc in #14016
• docs: update security support and roadmap after v2.3.0 by @phillipleblanc in #14024
• chore: post v2.3.0 release housekeeping by @bjchambers in #13969
• test(adbc): guard BigQuery corpus offline and in release gate by @phillipleblanc in #14017
• fix(duckdb): deny the regexp built-ins DuckDB cannot answer faithfully (fixes #13809) by @claudespice in #13871
• fix: Update tpch benchmark snapshots for federated/adbc[bigquery].yaml by @app/github-actions in #13984
• fix(search): drop the chunks a shortened row no longer produces from a chunked index (refs #13717) by @claudespice in #13960
• Reduce Cayenne allocations during primary-key validation and filtering by @lukekim in #14009
• test(forks): guard seven fork patches that had no repo-side test by @krinart in #13996
• fix(deps): bump arrow-rs to correctly-rounded Decimal→Float cast (closes #13978) by @Jeadie in #14012
• perf(vortex): defer projection setup on filtered scans until the filter resolves by @bjchambers in #14035
• endgame: include spiceai/skills versioned release by @lukekim in #14031
• fix(caching): partition doomed entries at the survivor cutoff so eviction converges (closes #13994) by @Jeadie in #14021
• fix(deps): bump arrow-rs fork pin for Decimal->Float rounding fix by @Jeadie in #14049
• Fix subqueries with use_source acceleration by @phillipleblanc in #14022
• fix(cayenne): make DELETE, UPDATE and INSERT work on a mode: memory acceleration (fixes #12008) by @bjchambers in #14047
• fix: clarify OpenDAL S3 retry warnings by @lukekim in #14040
• test(s3): run the parquet-overwrite fixtures on RustFS by @bjchambers in #14067
• feat(cayenne): materialize multi-reference CTEs on the query path by @lukekim in #13918
• perf(vortex): answer a constant IN list by probing a set, and falsify it by interval by @bjchambers in #14061
• perf(vortex): skip a scan split whose zones cannot satisfy the filter by @peasee in #14064
• fix(arrow): report an exact row count from the indexed point-lookup scan by @krinart in #13972
• fix(cayenne): apply sort_columns with refresh_mode: full by @peasee in #14063
• fix(smb): pad an empty CREATE buffer so Samba lists the share root (fixes #13293) by @grokspice in #14050
• perf(cache): key the logical-plan cache on SQL text, not parameter values by @bjchambers in #14069
• fix: harden HuggingFace E2E chat against slow Metal generation by @lukekim in #14072
• fix(cayenne): move accelerator filesystem I/O off Tokio workers by @lukekim in #14073
• test(chbench): enable CTE materialization and IVM on mysql/postgres adaptive HTAP by @lukekim in #14070
• ci: run Substrait Mode A TPC-H on pull requests and the merge queue by @lukekim in #14071
• feat(mcp): support MCP specification 2026-07-28 (dual-era) by @lukekim in #14043
• fix(ci): call a linker that died of a signal an infrastructure failure, not a check failure (fixes #13614) by @grokspice in #14044
• fix: Provide temporary directory in docker images by @Jeadie in #14089
• fix: restore OSS installer, CLI and test workflow coverage by @phillipleblanc in #14025
• Delete v2.2.0.md by @Jeadie in #14095
• docs(release): add v2.3.1 release notes by @phillipleblanc in #14094
• fix(test): allow DELETE in the CORS allow-methods assertion by @claudespice in #14098
• fix(ci): stop install-protoc unzipping into a shared ~/.local by @lukekim in #14097
• feat(connectors): add ORC listing format via in-repo FileFormat by @lukekim in #14075
• fix(cayenne): run snapshot bootstrap check before opening the metastore by @Jeadie in #14093
• feat(cache): verify the results-cache namespace prefix with Verus by @lukekim in #14074
• feat(cloud-connect): add a GetDatasets command that answers the /v1/datasets document (refs #13369) by @grokspice in #14051
• Suppress Cayenne startup logs when no Cayenne dataset is configured by @Jeadie in #14042
• fix(cache): re-bind parameter values when revalidating a stale result (fixes #14099) by @bjchambers in #14100
• fix(cluster): support distributed HTTP scans by @phillipleblanc in #14108
• fix(bigquery): keep ILIKE evaluation local by @phillipleblanc in #14110
• docs: update security support for v2.3.1 by @phillipleblanc in #14117
• feat(cayenne): reuse ScanView until write, lag only for read-only CDC by @lukekim in #14055
• fix(deps): remediate open Dependabot alerts by @phillipleblanc in #14111
• fix(testoperator): validate results in every scale factor 1 TPC-H, TPC-DS and ClickBench benchmark by @lukekim in #14119
• fix(cayenne): reject ambiguous metastore paths by @phillipleblanc in #14130
• perf: serve results-cache hits where the request arrives and cut per-hit overhead by @lukekim in #14103
• test(runtime): record query previews in the management export test by @lukekim in #14155
• ci: upgrade spiceio setup action to v0.11.0 by @lukekim in #14152
• feat(caching): Make caching_stale_if_error RFC-5861 compliant (with stale-if-error header) by @Jeadie in #14134
• perf(runtime-table): defer cache-eviction key extraction to entries a delete actually names by @Jeadie in #14138
• fix(runtime): report the acceleration.ready_state deprecation once per component (fixes #13749) by @claudespice in #14006
• fix(runtime): write the inferred Arrow sort order under the prefixed key its validation accepts (fixes #14023) by @claudespice in #14032
• fix(connectors): Fix JSON/Orca files using metadata columns by @Jeadie in #14115
• feat(cayenne): build secondary indexes from indexes in file and memory mode by @phillipleblanc in #14149
• fix(cayenne): round-trip decimal, binary, and time stats and drop them on scale change by @lukekim in #14139
• feat(cayenne): cluster warm and datalake tiers, and write full refreshes as key-range files by @lukekim in #14124
• fix(cayenne): compile the cold-tier pruning test and backtick a doc literal by @lukekim in #14175
• fix(cayenne): make the crates own targets lint and compile by @phillipleblanc in #14200
• test(forks): guard five more fork patches, and drop a row that is not fork state by @krinart in #14015
• perf(cache): promote encoded SQL results to raw after the second decode by @lukekim in #14199
• fix(turso): build a dictionary column directly so a dictionary over a list, map or boolean value reads back (fixes #13033) by @grokspice in #14181
• fix(ci): probe the macOS toolchain before reaching for brew in the release builds by @grokspice in #14203
• fix(ci): skip Metal kernel precompilation in the macOS release build by @grokspice in #14204
• fix(github): paginate nested GraphQL connections and pace to GitHub's rate limits by @lukekim in #14179
• fix(vortex): stop an IN list holding a NULL from panicking the scan by @krinart in #14163
• bench(cayenne): use std::hint::black_box in the clustering bench by @lukekim in #14129
• fix(runtime-table): stop rebuilding SessionContext on every cache fetch by @Jeadie in #14141
• test(chbench): cluster order_line, oorder and customer on the adaptive HTAP arms by @lukekim in #14192
• fix(runtime): count a first load as still loading in the Dataset load summary (fixes #13974) by @claudespice in #14020
• fix(duckdb): push regexp_count down again at a rendering that counts as the kernel does (fixes #13870) by @claudespice in #14153
• build: lint and test the sign-off under the same profile as the merge queue by @lukekim in #14180
• ci: require the Verus proofs in the merge queue as one check by @lukekim in #14189
• ci: run the longest macOS jobs on their own runner pool by @lukekim in #14229
• fix(cayenne): build the DELETE sink inside the execution-time write lock (fixes #13828) by @claudespice in #14218
• build(deps): bump the github-actions-dependencies group across 1 directory with 7 updates by @dependabot in #14231
• fix(cluster): decide what a Flight message carries by its IPC header, not its body length (refs #13737) by @claudespice in #14212
• ci: run Mode A TPC-H on merge queue and trunk/release push only by @lukekim in #14247
• chore(deps): bump spiceai/duckdb-rs to 76655d2f by @lukekim in #14246
• ci: stop exporting empty AWS and DuckLake endpoints to the schema test by @phillipleblanc in #14194
• fix(runtime): reload a localpod dataset when the dataset it reads through is reloaded (fixes #3288) by @claudespice in #14208
• build(deps): bump the aws-sdk group with 3 updates by @dependabot in #14255
• fix(ci): resolve Homebrew prefix when brew is the spice flock wrapper by @lukekim in #14210
• chore(deps): raise the datafusion-table-providers pin to include the NUMERIC result-column fix by @phillipleblanc in #14254
• ci: align the DuckLake bootstrap with the embedded DuckDB, wait for Databricks startup, and stop dispatching legs that cannot pass by @phillipleblanc in #14250
• fix(runtime): install the Spice function deny-list on the PostgreSQL catalog connector (refs #13664) by @claudespice in #14225
• perf(cayenne): share inline-cache view entries by Arc instead of cloning them per scan by @krinart in #14191
• build(deps): bump aws-actions/configure-aws-credentials by @dependabot in #14256
• ci: stop dispatching the indexed turso TPC-H SF1 tests by @phillipleblanc in #14252
• fix(catalog): keep the tables registered under an existing schema by @phillipleblanc in #14193
• feat(cache): Spice sharded cache as the sole LruCache engine by @lukekim in #14206
• ci: lint GitHub Actions definitions with actionlint, and fix the 73 findings it surfaced by @grokspice in #14223
• perf(cache): tighten the Raw SQL results-cache serve path by @lukekim in #14205
• ci: stop triggering the CUDA build on pull requests by @lukekim in #14267
• ci: run CodeQL on pull requests and the merge queue by @lukekim in #14269
• Release 2.3.2 release notes by @krinart in #14271
• chore: make AGENTS.md the canonical agent instructions by @lukekim in #14237
• ci: run CodeQL Analyze on spiceai-dev-runners by @lukekim in #14281
• feat: TypeSafe Jev System One evaluation provider by @lukekim in #14215
• fix(cayenne): tag file statistics bounds as the column's Arrow type (fixes #14280) by @phillipleblanc in #14283
• ci: install spiceio when the runner has no gh (refs #14233) by @lukekim in #14288
• test(forks): 11 repo guards by @krinart in #14261
• docs: add Spice.ai in Action manuscript and companion labs by @lukekim in #13965
• test(duckdb): add an integration test for the index CTE materialization by @sgrebnov in #13885
• fix(cayenne): coalesce inline writes into one batch by @sgrebnov in #14279
• release: Update SECURITY.md and endgame template after 2.3.2 by @peasee in #14293
• fix(cayenne): count each Arrow allocation once in the inline-cache gauge by @krinart in #14272
• feat(s3): SQS event-driven changes for refresh_mode: changes by @lukekim in #14121
• feat(caching): single-flight coalesce concurrent cache-miss fetches by @Jeadie in #14142
• fix(caching): make caching_stale_if_error detect transient HTTP failures on real schemas by @krinart in #14161
• Use Cayenne secondary indexes for dynamic join filters by @phillipleblanc in #14282
• fix(ci): compare DynamoDB sets without relying on element order by @bjchambers in #14328
• docs(release): Remove QA analytics step from endgame by @peasee in #14329
• ci: run CodeQL on the merge queue and trunk, not on pull requests by @lukekim in #14290
• docs(cayenne): reposition the reference, add query serving, re-audit against trunk by @lukekim in #14338
• docs(endgame): update versioned docs release steps by @ewgenius in #14277
• test(forks): guard the ballista per-task file-scan restriction by @krinart in #14292
• fix(cli): surface the full error chain for spice chat connection failures by @krinart in #14289
• fix(ci): degrade the incomplete-sign-off handler when the runner has no gh (refs #14234) by @claudespice in #14304
• fix(runtime-table): serialize a direct write against acceleration snapshot creation (fixes #13548) by @claudespice in #14310
• fix(duckdb): screen regexp_like and regexp_replace as regexp_count is screened (fixes #14148) by @claudespice in #14321
• fix(http): honor retry budget without an extra origin request by @phillipleblanc in #14313
• Fix SchemaCastScanExec's schema conversion in fn partition_statistics by @Jeadie in #14258
• fix(cluster): recognise a Flight keepalive by its empty envelope, not by what its header declares (fixes #13737) by @claudespice in #14327
• perf(http): defer zero-TTL acceleration lookup until origin failure by @phillipleblanc in #14302
• fix(cayenne): keep a key visible when it is re-inserted over a stale-insert tombstone by @sgrebnov in #14312
• perf(cayenne): read only key and filter columns in filtered key deletes by @sgrebnov in #14374
• fix(cayenne): let small protected-snapshot merges run during a long large-tier merge by @sgrebnov in #14296
• perf(cayenne): serve primary-key lookups on a freshly loaded table in ~1 ms by @lukekim in #14314
• fix(cayenne): skip min/max statistics for nested columns (fixes #14368) by @sgrebnov in #14392
• test(caching): cover SchemaCastScanExec statistics projection by name, retype, and SQL filter by @Jeadie in #14146
• fix(turso): keep a quantified comparison out of Turso SQL (fixes #14041) by @grokspice in #14393
• fix(udfs): declare the local_embed dev-dependency the embed tests need (fixes #13092) by @claudespice in #14377
• perf(cayenne): batch metastore manifest rewrites, upsert in place, and run every write on one writer connection by @lukekim in #14369
• fix(runtime-table): ignore zero-row batches in stale fallback by @phillipleblanc in #14331
• Prune object-store file listing by _last_modified predicates by @Jeadie in #14265
• Reading only partition or metadata columns needlessly scans all file contents by @Jeadie in #14116
• fix(duckdb): keep a concat over a binary operand out of the federated plan (fixes #13915) by @claudespice in #14333
• fix(ci): keep the sign-off attribution inside GitHub's 140-character status cap (fixes #14076) by @grokspice in #14390
• fix(ci): expose a present-but-unlinked cc tool on macOS runners instead of routing it through brew install (fixes #13479) by @grokspice in #14387
• ci: re-measure integration.yml's job bounds after the archive consolidation (fixes #13429) by @grokspice in #14388
• fix(ci): start DuckLake's local MinIO from an image that is still published by @grokspice in #14399
• test(runtime-table): make the metric-scraping refresh tests pass under cargo test by @claudespice in #14381
• fix(federation): keep a correlated subquery predicate above a join of two sources (refs #8220) by @claudespice in #14372
• fix(caching): snapshot staleness before the origin fetch for stale_if_error by @Jeadie in #14263
• perf(snapshots): skip unchanged snapshot metadata with a conditional GET by @sgrebnov in #14409
• fix(search): prune the rest of a key group from an Elasticsearch chunked index (refs #13717) by @claudespice in #14320
• fix(bench): derive MySQL's empty-field NULL handling from the column type (refs #13152) by @claudespice in #14345
• fix(graphql): debit a LIMIT by the rows a page returned, not the declared page size (fixes #14308) by @claudespice in #14353
• fix(ci): fit retention_oom's retry budget inside its workflow step, and guard the coupling (fixes #13512) by @grokspice in #14389
• docs: say plainly what Spice is, refresh the README for v2.3, and promote connector statuses by @lukekim in #14410
• fix(ci): pin, checksum and retry the oha download in the E2E graceful-shutdown jobs by @grokspice in #14418
• fix(snapshots): resolve snapshot entries relative to the metadata location (#14425) by @sgrebnov in #14426
• Lower the GitHub connector default concurrency limit to 4 by @lukekim in #14431
• build(deps): bump nvidia/cuda in the docker-dependencies group by @dependabot in #14439
• build(deps): bump the aws-sdk group with 3 updates by @dependabot in #14440
• build(deps): bump the github-actions-dependencies group across 1 directory with 5 updates by @dependabot in #14441
• test(cayenne): bound the refused-build guard by builds, not by the host's speed by @grokspice in #14424
• fix(cache): don't report an invalidation cancelled by runtime shutdown as a failure by @grokspice in #14417
• ci: run remote sign-off on the spiceai-macos pool by @lukekim in #14449
• ci: run CodeQL Analyze on spiceai-macos by @lukekim in #14442
• fix(runtime): keep the built-in date_part so both weekday spellings agree (fixes #13920) by @claudespice in #14154
• fix(cayenne): include the in-memory CDC tier when an overwrite or a retention pass covers the whole table by @lukekim in #14428
• fix: keep NOT IN null-aware through the join reorder and the Cayenne sort-merge rewrite by @lukekim in #14429
• fix(cayenne): discard a compaction whose snapshot an overwrite replaced mid-pass by @lukekim in #14432
• fix(cayenne): stop maintained views, Vortex IN lists and dynamic-filter sharing from returning wrong rows by @lukekim in #14427
• fix(cayenne): hide spilled rows on CDC upsert fallback by @bjchambers in #14416
• fix(ci): skip the integration, ADBC, chDB and E2E gate jobs on pull requests instead of passing them (fixes #13841) by @grokspice in #14454
• fix(cayenne): judge a filtered key delete's captured sources by the index captured with them (refs #13913) by @claudespice in #14455
• fix(ci): clear the macos-15 image's openssl@1.1 symlink before installing MySQL by @lukekim in #14489
• ci: produce CodeQL SARIF in the Analyze job on spiceai-macos by @lukekim in #14500
• fix(cayenne): correctness fixes for the goal-driven adaptive controller, with a closed-loop simulation harness by @lukekim in #14443
• fix(cdc): keep the newest source commit timestamp on a coalesced change batch by @lukekim in #14463
• fix(cayenne): draw a sequence for a current-snapshot append so per-key OCC can order it (fixes #13685) by @claudespice in #14360
• fix(http): name a configured model's load failure instead of reporting it not found (fixes #13303) by @claudespice in #14395
• fix(duckdb): keep inferred source indexes off change-stream accelerations so upserts commit under concurrent reads (refs #13929) by @claudespice in #14396
• fix(duckdb): keep a text cast over a binary operand out of the federated plan (fixes #14355) by @claudespice in #14448
• fix(llms): honor tool_choice required and allowed_tools on mistral.rs-hosted models instead of panicking (fixes #14230) by @claudespice in #14460
• test(runtime): run the load-error counter test in its own process (fixes #13085) by @claudespice in #14462
• fix(runtime): stop a replaced dataset configuration's load from registering over the new one (fixes #1458) by @claudespice in #14367
• fix(install): stop asking for sudo on a first install into a fresh HOME (fixes #14445) by @claudespice in #14495
• perf(cayenne): serve primary-key point lookups in half the time by @lukekim in #14433
• fix(deps): bump DataFusion for upstream fixes to wrong results from filter pushdown, simplification and planning by @lukekim in #14430
• fix(runtime): size every internal DataFusion session from the CPU budget by @bjchambers in #14412
• fix(runtime): serve nested, zoned and half-float columns from the Iceberg catalog API (fixes #4815) by @claudespice in #14480
• fix(acceleration): keep cached results when a snapshot refresh finds no newer snapshot by @sgrebnov in #14497
• fix(runtime): sleep cron tests to the next boundary, not one that already fired (fixes #13759) by @claudespice in #14474
• fix(ci): derive every lint-rust guard make runs, whatever its separator or recipe layout (fixes #13783) by @claudespice in #14475
• fix(cayenne): stop the small-file compaction of a position-mode PK table from deadlocking on its own write lock (fixes #14420) by @claudespice in #14481
• fix(runtime-table): report refresh bytes for the rows each batch holds by @Jeadie in #14469
• fix(spark,databricks): keep Spice-only functions out of SQL sent to Spark Connect and Databricks SQL Warehouse (refs #13664) by @claudespice in #14498
• fix(vortex): size a cached footer by what it retains, not its serialized bytes (fixes #12917) by @claudespice in #14502
• fix(cayenne,telemetry): Register accelerated sink dataset immediately from existing acceleration by @peasee in #13955
• Update spicepod.yml by @Jeadie in #14533
• fix(cayenne): keep a rewrite's count inexact when it retains a late protected snapshot (fixes #14383) by @claudespice in #14385
• fix: Update tpch benchmark snapshots for accelerated/on_zero_results/file[parquet]-cayenne[file]-on_zero_results.yaml by @app/github-actions in #14408
• build(rust): upgrade toolchain to 1.98.1 by @lukekim in #14560
• fix(postgres): release a shared slot's hold on a table its publication cannot drop (fixes #13032) by @claudespice in #14527
• fix(cayenne): size the build side before sort-merging a non-Cayenne outer join by @krinart in #14520
• Update DF Upgrade template by @krinart in #14526
• fix(ci): wait for the refresh to invalidate the results cache, not a fixed 3s by @grokspice in #14598
• fix(deps): move the DataFusion pin past the four unparser fixes, and guard each of them (fixes #13022) by @grokspice in #14570
• fix(dynamodb, cosmosdb, mongodb): push filters down only where the source evaluates them as SQL does by @lukekim in #14419
• feat(snapshots): serve a dataset from published snapshots with from: s3://… and file_format: snapshot by @lukekim in #14529
• perf(cayenne): keep the per-shard PK index across checkpoint flushes and back off futile bakes; fix q17 and two wrong-results races (fixes #14235) by @lukekim in #14555
• test(chbench): lower the MySQL adaptive pods' CDC linger to 500 ms by @lukekim in #14615
• fix(graphql): show the parse-failure bytes in JSON decode previews by @Jeadie in #14530
• Defer source-first cache fallback planning by @phillipleblanc in #14556
• perf(cayenne): order whole-table rewrites per scan partition, then merge by @bjchambers in #14488
• fix(ci): tolerate the parquet-rename race's third DuckDB error in the E2E log scan by @grokspice in #14541
• fix(graphql): retry an inferred credential refusal instead of failing the refresh by @Jeadie in #14539
• fix(cayenne): serve a widened table's files from their persisted statistics (refs #13829) by @claudespice in #14316
• fix(acceleration): checkpoint DuckDB's write-ahead log before a snapshot copies its file (fixes #13912) by @claudespice in #14325
• fix(ci): let E2E cleanup run when the job failed before creating its working directory by @grokspice in #14553
• Replace unmaintained backoff with a workspace crate by @phillipleblanc in #14621
• test: read Expect stand-ins with the installed shell by @phillipleblanc in #14634
• fix(models): apply a tool_choice that forces a call to one round of the tool-use loop (fixes #14459) by @claudespice in #14635
• fix(runtime): don't end a tool-use stream on a stale tool_calls finish (fixes #13309) by @claudespice in #14548
• fix(smb): name listed locations under the share so directory datasets load (fixes #14060) by @claudespice in #14549
• fix(models): name a configured model's load failure in ai() and streaming nsql (fixes #14394) by @claudespice in #14632
• test(chbench): remove the mysql-cayenne[file]-adaptive-split HTAP arm by @sgrebnov in #14620
• fix(cayenne): record sharded CDC keys in the table-wide PK index by @lukekim in #14603
• fix(sqlite): keep TRY_CAST and every cast SQLite answers differently out of the federated plan (fixes #14398) by @claudespice in #14496
• ci: move remaining Ubuntu 22.04 references to 24.04 by @lukekim in #14677
• fix(snapshots): retry a failed snapshot attempt with backoff by @sgrebnov in #14676
• fix(snapshots): large snapshots no longer fail bootstrap on slow connection by @sgrebnov in #14571
• fix(refresh): start a refresh's source scan only when the sink reads it, so the full-text index keeps its rows (fixes #14619) by @claudespice in #14663
• test(cayenne): compare suite answers cell by cell against SQLite and chDB by @lukekim in #14473
• Evaluate any chat model via /v1/evaluate by @lukekim in #14568
• ci: quarantine the management API integration schedule until its dev OAuth client is reactivated (refs #12376) by @grokspice in #14684
• ci: stop scheduled Testoperator Ballista benchmarks by @phillipleblanc in #14682
• fix(cayenne): stop snapshotting datasets with a datalake tier, whose restored copies deleted each other's files by @lukekim in #14583
• fix(cayenne): count each Arrow allocation once for the mem-tier limit and checkpoint write sizing by @krinart in #14675
• fix(ci): classify a sign-off that reached its step budget as signalled, so an expiry publishes no verdict (fixes #13843) by @grokspice in #14685
• fix(runtime): retry a dataset whose source is unreachable at startup (fixes #14609) by @bjchambers in #14623
• fix(cayenne): archive only the snapshot directories a dataset snapshot references (fixes #14605) by @sgrebnov in #14627
• ci: route trunk-push macOS builds to the standard pool by @phillipleblanc in #14645
• ci: install strip for the retention OOM regression test by @phillipleblanc in #14701
• fix(smb): serve every share on a host from the one store registered for it (fixes #14550) by @grokspice in #14689
• fix(runtime): discard cached plans and results when a dataset is unloaded (refs #14251) by @claudespice in #14365
• feat(snapshots): reload snapshot-mode datasets from S3 event notifications (SQS) by @lukekim in #14335
• fix(cayenne): report runs per size tier when protected-snapshot compaction declines (fixes #13622) by @claudespice in #14476
• fix(cayenne): re-run a post-write compaction pass a concurrent append asked for (fixes #13906) by @claudespice in #14479
• fix(http): keep the request URL out of HTTP connector errors (fixes #13534) by @claudespice in #14490
• fix(runtime): load every localpod dataset that reads from one parent at startup (fixes #13087) by @claudespice in #14532
• fix(build): share one cargo metadata helper across the lint guards, so a broken cargo is never a violation (fixes #13121) by @claudespice in #14534
• fix(duckdb): push regexp_replace down again for a one-digit group reference, and refresh the Postgres ClickBench q35 plan (refs #13966) by @claudespice in #14552
• fix(duckdb): keep a cast into binary out of the federated plan (refs #14397) by @claudespice in #14633
• fix(runtime): keep finished async query jobs finished, and delete expired ones by @lukekim in #14585
• fix(federation): keep DataFusion's cast built-ins out of every federated plan (fixes #14444) by @claudespice in #14484
• test(cayenne): cover a swapped join filter through the sort-merge rewrite end to end (refs #14235) by @claudespice in #14551
• fix(runtime): decide every Spark/built-in function collision by name, and refuse an undecided one (fixes #14361) by @grokspice in #14400
• fix(ci): run every bin target's unit tests in the sign-off gate, and give spice connect its own --cloud-region refusal (fixes #13426) by @grokspice in #14406
• test(data_components): compile the federation unparser guards in every scoped test run (fixes #13625) by @grokspice in #14447
• fix(runtime-component): skip an inferred Cayenne index on a floating-point column instead of failing the load (fixes #14590) by @grokspice in #14599
• fix(cache): serve stale cached results during frequent table updates (stale_while_revalidate_ttl) by @sgrebnov in #14708
• Update Turso to 0.8.1 and retry write conflicts a metastore statement raises by @lukekim in #14680
• fix(snapshots): stop a snapshot dataset's load when a reload replaces it by @lukekim in #14673
• fix(cayenne): hand non-partition filters to every partition scan (fixes #12959) by @claudespice in #14370
• fix(postgres-accel): resolve secret references in the sidecar connection parameters (fixes #13296) by @claudespice in #14544
• fix(cayenne): drain the in-memory CDC tier before a full rewrite scans it (fixes #14450) by @claudespice in #14486
• feat(caching): warm SQL results cache on first refresh from persisted plan shapes by @lukekim in #14178
• fix(ci): retry only the container startup in the zero-retry MySQL CDC tests by @grokspice in #14727
• feat(key-index): immutable secondary index runs over compound Arrow keys by @bjchambers in #14592
• fix(federation): keep a fractional-to-integer cast out of the plans pushed to DuckDB, PostgreSQL, MySQL and BigQuery (fixes #14482) by @grokspice in #14601
• fix(testoperator): give accelerated bench configs a 300s ready_wait and name unready datasets on timeout (fixes #13973) by @claudespice in #14728
• fix(federation): keep arrow_typeof and its plan-introspection siblings local on every backend (fixes #14334) by @grokspice in #14695
• test(bench): refresh the tpch_q16 explain snapshots for the null-aware NOT IN plan (fixes #13977) by @claudespice in #14731
• ci: build trunk-push macOS legs on spiceai-macos-large again, keeping one-at-a-time coalescing by @grokspice in #14735
• ci(verus): derive the verified crates from cargo metadata; pin vstd once by @bjchambers in #14709
• docs: raise the test and evidence bar: differential first, exact assertions, performance always measured by @lukekim in #14723
• test(vortex): wait for a dropped segment cache to be freed before asserting it is gone (fixes #13295) by @claudespice in #14470
• ci(integration): report each integration part's own result in its required check by @grokspice in #14743
• fix(cayenne): back off futile bakes under a violated query goal too by @lukekim in #14736
• feat(object_store_occ): add transactional WAL and MVCC snapshots by @lukekim in #14732
• fix(snapshots): bootstrap readers from late snapshot publications by @phillipleblanc in #14468
• fix(cayenne): apply retention_sql to a mode: memory acceleration (fixes #14045) by @claudespice in #14344
• fix(duckdb, sqlite): keep the first copy of a key a write repeats under on_conflict: drop (fixes #14629) by @claudespice in #14748
• fix(cache): release a removed dataset's memory when the plan cache discards its plans (fixes #14251) by @claudespice in #14760
• build(deps): bump the aws-sdk group with 2 updates by @dependabot in #14764
• fix(llms): explain an Anthropic model's refusal of a sampling control instead of passing the bare 400 through (fixes #13564) by @grokspice in #14700
• build(deps-dev): bump dompurify by @dependabot in #14613
• fix(snapshots): allow cayenne_file_path and cayenne_metadata_dir on file_format: snapshot datasets (fixes #14696) by @sgrebnov in #14698
• test(snapshots): make snapshot integration tests robust under parallel CI runs by @sgrebnov in #14765
• feat(cayenne): keep the secondary index current across every write by @bjchambers in #14593
• fix: Update tpch benchmark snapshots for accelerated/file[parquet]-duckdb[memory].yaml by @app/github-actions in #14674
• Isolate Docker integration fixtures across concurrent processes by @phillipleblanc in #14639
• ci: run Substrait Mode A TPC-H at SF 1 on spiceai-macos in every merge-queue entry by @lukekim in #14522
• ci: stop repeating scheduled benchmarks: one source per accelerator, hosted sources weekly, a release commit once by @lukekim in #14681
• build(deps): bump nvidia/cuda by @dependabot in #14763
• fix(http): stop a non-2xx response body replacing an accelerated table's rows (fixes #13515) by @claudespice in #13538
• fix(chat-api): answer a tool-call-only assistant turn instead of panicking (fixes #13207) by @grokspice in #14232
• fix(sqlite, duckdb): keep a decimal AVG, and on SQLite a decimal SUM, out of the federated plan (fixes #14492) by @claudespice in #14670
• fix(spiceai, duckdb, sharepoint): name the Spicepod key a missing-parameter error asks for (refs #14446) by @claudespice in #14769
• Add table-bound ChangeSink ownership and backends by @phillipleblanc in #14704
• fix(ci): serialize rustup installs on shared macOS runners by @lukekim in #14717
• fix(cayenne): keep the per-shard PK index when the table-wide index is discarded by @lukekim in #14604
• Upgrade to DataFusion 55.1 and Arrow 59.3 by @krinart in #14612
• Update openapi.json by @app/github-actions in #14742
• build(deps): bump rustls from 0.23.40 to 0.23.45 by @dependabot in #14773
• Upgrade to iceberg-rust to 0.11.0 and DF 55.2 by @sgrebnov in #14771
• fix(cayenne): write an append into an empty unkeyed table as one load by @phillipleblanc in #14784
• fix(mysql_replication): don't re-send a live member's delivered commits after a reconnect by @lukekim in #14751
• ci(codeql): don't fail the SARIF upload when the merge queue already deleted its ref by @grokspice in #14793
• fix(acceleration): accept time_format iso8601 when the time_column is already a timestamp by @lukekim in #14777
• fix(cache): store SQL results that go stale mid-query by @lukekim in #14710
• fix(cayenne): serve filtered and global maintained aggregates by @lukekim in #14761
• perf(cayenne): build the checkpoint's tombstone union after releasing the capture locks by @lukekim in #14759
• ci: keep no artifacts or caches from PR and merge-queue checks, and run gating macOS jobs on spiceai-macos-large by @lukekim in #14804
• fix(cayenne): keep maintenance from deleting files a snapshot is archiving by @sgrebnov in #14789
• fix(snapshots): never overwrite shared snapshot metadata, and publish more than once to a file:// location by @lukekim in #14582
• perf(cdc): build deferred change rows in the reader while the apply runs by @lukekim in #14746
• fix: Update tpch benchmark snapshots for accelerated/on_zero_results/file[parquet]-cayenne[file]-on_zero_results.yaml by @app/github-actions in #14786
• test(testoperator): add a cold-start time-to-ready regression test by @phillipleblanc in #14785
• test(testoperator): make append tests more robust by @sgrebnov in #14817
• fix(runtime,data_components): wait for change-data-capture sources to record their positions before shutdown closes the accelerations (fixes #14523) by @grokspice in #14702
• fix(graphql): detect non-JSON responses and treat gateway errors as retryable by @lukekim in #14781
• fix(runtime): require keys for s3_auth: key, honor gs:// state location params, and leave newer rate-control state alone by @lukekim in #14584
• build(deps): bump hickory-resolver from 0.26.1 to 0.26.2 by @dependabot in #14774
• fix(listing): skip zero-byte objects (S3 folder markers) in format-selected listings by @sgrebnov in #14822
• Avoid unnecessary repartitioning in indexed dynamic-filter joins by @bjchambers in #14799
• feat(cayenne): keep one row per primary key automatically and deprecate on_conflict by @bjchambers in #14726
• fix(object-store): stop reading a response body at its first error by @phillipleblanc in #14831
• fix(ci): use byte-order collation in the benchmark Postgres container (refs #14815) by @sgrebnov in #14836
• fix(listing): keep a partition predicate as a residual filter on the _location fast path by @grokspice in #14790
• fix(ci): search only the unixodbc keg, not all of Homebrew's lib, in the macOS release builds by @grokspice in #14846
• fix(federation): keep date, timestamp and interval values local on SQLite reached through ADBC or ODBC (fixes #14753) by @claudespice in #14840
• chore: pin the fork at spiceai/datafusion#249 and guard the DataFusion fixes backported from 54 by @krinart in #14800
• Revert "fix(sqlite, duckdb): keep a decimal AVG, and on SQLite a decimal SUM, out of the federated plan (fixes #14492) (#14670)" by @krinart in #14825
• perf(runtime): skip EnsureRequirements while planning a point lookup by @lukekim in #14807
• fix(testoperator): report CH-benCH queries with no rows on either side as vacuous, not as matches by @lukekim in #14810
• feat(acceleration): make Cayenne the default acceleration engine by @phillipleblanc in #14837
• fix(cayenne): keep a join's LIMIT when the oversized-join rewrite makes it a sort-merge join by @lukekim in #14805
• test(bigquery): exclude subtrees an empty join build side never runs from the corpus job count (fixes #14848) by @sgrebnov in #14849
• fix(cayenne): fail contradictory write settings once, and order NULL times below every time by @bjchambers in #14847
• fix(runtime): serve an existing acceleration while its source is unavailable (fixes #14610) by @bjchambers in #14624
• Prune object-store file listing by metadata columns (#14264) by @Jeadie in #14303
• fix(vortex): fold partition values into the file-pruning predicate; test null-equal joins under mode:file by @lukekim in #14797
• feat(rate-control): adaptive, bounded and cluster-coordinated HTTP rate controls by @Jeadie in #14143
• fix(sql): align date_part('dow') with EXTRACT(dow) Sunday=0 by @lukekim in #14796
• fix(runtime): carry API-key principal into MCP tools/call by @lukekim in #14828
• perf(cayenne): hold maintained-aggregate retraction state in a compact shared index by @lukekim in #14762
• fix(deps): keep a hash join's order once it has a dynamic filter, so vector_search plans again by @Jeadie in #14857
• fix(cayenne): fence schema statistics and control maintenance tests by @bjchambers in #14856
• fix(github): retry nested GraphQL pages in place and fail incomplete nested connections by @sgrebnov in #14862
• fix(cayenne): load an append dataset that has retention_sql and orders versions by time by @phillipleblanc in #14878
• fix(cdc): Cayenne replication needs only a primary key, not on_conflict by @phillipleblanc in #14880
• feat(connector-huggingface): Hugging Face datasets data connector (hf://datasets/...) by @lukekim in #14877
• fix(iceberg): Iceberg REST clients read the real table or get an error, never an empty one by @lukekim in #14588

Full Changelog: https://github.com/spiceai/spiceai/compare/v2.3.2...v2.4.0-rc.1

Spice v2.3.2 is now available! ⚡

Spice v2.3.2 makes point lookups and repeated queries faster with Spice Cayenne. SQL results-cache improvements apply to every cached query, regardless of its data source or accelerator. Across two benchmark rounds, the time for a small cache hit fell by 50-52% over HTTP and 44-48% over Flight SQL. For small cache hits, CPU time per server request fell by 58-61% over HTTP and 49-51% over Flight SQL.

Highlights in v2.3.2 include:

• Indexed Point Lookups — Cayenne now honors indexes, so a lookup reads the matching rows instead of scanning
• Cayenne Data Layout — cayenne_cluster_by groups related rows together across storage tiers
• Faster Queries After a Full Refresh — refreshed tables are written so that filtered queries read fewer files
• Faster Cached Responses — these improvements apply to every query in the SQL results cache, regardless of its data source or accelerator. For small results, cache-hit time fell by 44-52%
• Faster Repeat Queries — Cayenne reuses its prepared view of a table until the data changes
• Localpod Fix — a localpod dataset no longer serves data its source has replaced
• Catalog Fix — tables no longer go missing when datasets load at the same time
• Safer Cayenne Configuration — an ambiguous metastore configuration is reported instead of appearing as lost data
• Distributed HTTP Queries — async distributed queries can read HTTP datasets

What's New in v2.3.2​

Faster Point Lookups with Cayenne Indexes​

Every other accelerator already accepted indexes, and Cayenne logged that it ignored them. Cayenne now builds an index for each entry, in both mode: file and mode: memory:

acceleration:
  engine: cayenne
  mode: file # or memory
  indexes:
    '(TenantId, ServiceId)': enabled

A query that pins every column of an index to a value, such as WHERE TenantId = 7 AND ServiceId = 'a', now reads the matching rows directly. Without an index, Cayenne can only skip files whose minimum and maximum values rule the key out, which rarely helps when related rows are spread across the table — a lookup on a 5.4M-row test dataset had to open about half its files for nearly every key.

Indexes only reduce what a query reads, so results are identical either way. A query that uses a range, an IN list, an OR, or a cast on the indexed column reads the table as before. Index definitions are not stored with the table, so adding or removing one takes effect the next time the dataset loads. Floating-point columns cannot be indexed and are reported at load time.

EXPLAIN shows whether a query used an index, and the cayenne_lookup_index_probe_total metric counts lookups by outcome.

Control How Cayenne Lays Out Data​

cayenne_cluster_by stores rows with similar values near each other, so a filtered query reads fewer files. It applies to every storage tier, replacing cayenne_datalake_clustering_columns, which affected only the coldest tier.

acceleration:
  engine: cayenne
  params:
    cayenne_cluster_by: 'tenant_id, event_time'

CREATE TABLE ... CLUSTER BY (column, ...) is also supported, and several cases it previously rejected — a single column in parentheses, and names whose capitalization differs from the column definition — now work. A column name that does not exist is reported when the dataset loads rather than failing later.

Faster Queries After a Full Refresh​

A full refresh previously spread each key across every file it wrote, so a filtered query had to open all of them even when it wanted a single row. A refreshed table is now written so that each file holds a distinct range of the data, and queries filtering on that range read only the files that can match.

This needs no configuration and applies to any dataset whose accelerated table is replaced by a refresh. Datasets that already set cayenne_sort_columns or cayenne_cluster_by keep their existing layout, and the very first load is unchanged.

Faster Cached Query Responses​

Before this release, the runtime planned a query before it checked the SQL results cache. It also repeated other work that a cache hit does not need. The runtime now checks the cache first and returns cached answers directly. These improvements apply to every cached query, regardless of its data source or accelerator.

Across rounds of benchmarking queries that returned a seven-row GROUP BY result, cache-hit time over HTTP fell by 50-52%. Over Flight SQL, cache-hit time fell by 44-48%. In one HTTP round, cache-hit time fell from 63.8 µs to 30.8 µs. In one round over Flight SQL, it fell from 55.3 µs to 30.9 µs.

With task history enabled, the server used 125 µs of CPU time for a small HTTP hit before the change. It used 49 µs after the change. For a small result over Flight SQL, the server used 330 µs before the change. It used 168 µs after the change. These changes reduced CPU time by 61% and 49%, respectively.

For a wide result over HTTP, the server used 241 µs of CPU time before the change. It used 116 µs after the change. This is a 52% reduction. Wide results did not reduce the runtime's cache-hit time in those runs. Some measurements were slower.

Raw cache entries now share each stored batch with streaming JSON HTTP and Flight SQL responses. Buffered HTTP formats — including CSV, plain text, vnd.* envelopes, and JSON with union columns — still clone batches while encoding.

A stream benchmark with eight 20-column batches fell from 1.8722 µs to 226.23 ns. This is an 8.3x speedup. One 200-column batch fell from 2.0190 µs to 131.94 ns. This is a 15x speedup. These figures measure the construction and full consumption of the stream. They do not measure query latency from start to finish.

For compressed entries, the first read keeps the entry compressed. The second read promotes it to raw data when the raw size fits the cache. Later reads avoid decompression. In a cache benchmark, a later read took 17.1-17.8 µs across three payload sizes. A decode took 39.1-154.8 µs. Cached answers, expiry times, and cache settings are unchanged.

Faster Repeat Queries on Cayenne​

Cayenne prepares a view of a table's current contents before it can answer a query. It now reuses that preparation until the data actually changes, instead of rebuilding it, so repeated queries against a table that is not being written return faster and stop querying the metastore entirely once warm.

Datasets fed by continuous change data capture — a cdc: or debezium: source, or refresh_mode: changes — reuse the preparation for up to one second so a burst of incoming changes can share it. Datasets that accept writes always see their own writes immediately. No configuration is required.

Better Pruning for Decimal, Binary, and Time Columns​

Cayenne did not record minimum and maximum values for decimal, binary, time, and 16-bit float columns, so queries filtering on them could not skip files and had to read more data than necessary. Those columns now carry the same statistics as every other type, and clustering on a decimal column works as intended.

After a change that widens a decimal column's scale, Cayenne discards the affected statistics and rebuilds them, so queries read a little more until that completes. Results are unaffected.

Localpod Datasets Stay in Sync with Their Source​

A localpod dataset reads through another dataset in the same Spicepod. When the source dataset was reloaded — because its configuration changed — the localpod dataset kept reading the replaced copy, so it answered with data the source no longer had, and both copies kept refreshing.

A localpod dataset now reloads whenever the dataset it reads through does, including through several levels of chaining, and its cached results are cleared at the same time. This also works when the source is named with its full path, such as localpod:spice.public.parent. Fixes #3288.

Catalog Tables No Longer Go Missing​

When two datasets in the same schema loaded at the same time, one could be silently discarded, and every later query against it failed with Table not found. Both datasets reported that they had loaded successfully, and which one went missing varied between restarts. Datasets that share a schema now always both register.

Cayenne Detects an Ambiguous Metastore at Startup​

Cayenne datasets that each set a different cayenne_file_path, without a shared cayenne_metadata_dir, could open the wrong metadata directory after a restart. Cayenne then started up empty even though the data was still on disk, which looks like a total loss of accelerated data.

This configuration is now rejected at startup, naming each dataset and path involved and linking to the documentation. If a Spicepod uses several Cayenne data paths, set the same cayenne_metadata_dir on each dataset before upgrading.

Distributed Queries over HTTP Datasets​

An async distributed query submitted to /v1/queries failed if it read an unaccelerated HTTP dataset. These queries now run, with credentials, headers, and pagination behaving as they do for a non-distributed query. Fixes #14104.

Other Fixes​

• Partitioned datasets: a full refresh of a partitioned dataset only replaced the partitions the new data reached, so rows deleted at the source stayed queryable, and a refresh that returned no rows changed nothing. Every partition is now replaced.
• Iceberg write-through: a write to a partitioned dataset could deadlock against a refresh running at the same time, leaving both waiting.
• BigQuery: a case-insensitive LIKE could fail the query or return the wrong rows, because BigQuery has no ILIKE. Spice now evaluates it locally. Ordinary LIKE is unchanged.
• PostgreSQL: timestamps written back to PostgreSQL lost everything below the second. Microseconds are now preserved.

Dependency Updates​

No crate versions changed in this release. DataFusion remains at v54.1.0, Arrow remains at v58.3.0, and Vortex remains at v0.79.0.

Spice updates two fork revisions: datafusion-table-providers for the BigQuery and PostgreSQL fixes above, and duckdb-rs so the bundled DuckDB builds against the macOS 27 SDK.

Contributors​

• @lukekim
• @phillipleblanc

Breaking Changes​

cayenne_datalake_clustering_columns is replaced by cayenne_cluster_by. Rename the parameter before upgrading. The new one groups data on every storage tier, not only the coldest:

acceleration:
  engine: cayenne
  params:
    cayenne_cluster_by: 'tenant_id, event_time'

A dataset that sets both cayenne_sort_columns and a cluster key is now rejected when it loads. Remove cayenne_sort_columns to keep the cluster key.

Cayenne datasets using several data paths must share a metastore. If your Cayenne datasets set different cayenne_file_path values, set the same cayenne_metadata_dir on each one before upgrading. Spice now refuses to start on this configuration instead of risking an empty-looking acceleration:

acceleration:
  engine: cayenne
  params:
    cayenne_file_path: /mnt/a/cayenne
    cayenne_metadata_dir: /mnt/shared/cayenne-metadata

Cookbook Updates​

No new cookbook recipes.

The Spice Cookbook includes more than 104 recipes to help you get started with Spice quickly and easily.

Upgrading​

To upgrade to v2.3.2, use one of the following methods:

CLI:

spice upgrade

Homebrew:

brew upgrade spiceai/spiceai/spice

Docker:

Pull the spiceai/spiceai:2.3.2 image:

docker pull spiceai/spiceai:2.3.2

For available tags, see DockerHub.

Helm:

helm repo update
helm upgrade spiceai spiceai/spiceai --version 2.3.2

AWS Marketplace:

Spice is available in the AWS Marketplace.

What's Changed​

Changelog​

• fix(postgres): preserve microseconds in timestamp writeback by @phillipleblanc in #13963
• feat(cayenne): reuse ScanView until write, lag only for read-only CDC by @lukekim in #14055
• perf: serve results-cache hits where the request arrives and cut per-hit overhead by @lukekim in #14103
• fix(cluster): support distributed HTTP scans by @phillipleblanc in #14108
• fix(bigquery): keep ILIKE evaluation local by @phillipleblanc in #14110
• feat(cayenne): cluster warm and datalake tiers, and write full refreshes as key-range files by @lukekim in #14124
• fix(cayenne): reject ambiguous metastore paths by @phillipleblanc in #14130
• fix(cayenne): round-trip decimal, binary, and time stats and drop them on scale change by @lukekim in #14139
• feat(cayenne): build secondary indexes from indexes in file and memory mode by @phillipleblanc in #14149
• fix(catalog): keep the tables registered under an existing schema by @phillipleblanc in #14193
• ci: stop exporting empty AWS and DuckLake endpoints to the schema test by @phillipleblanc in #14194
• perf(cache): promote encoded SQL results to raw after the second decode by @lukekim in #14199
• perf(cache): tighten the Raw SQL results-cache serve path by @lukekim in #14205
• fix(runtime): reload a localpod dataset when the dataset it reads through is reloaded (fixes #3288) by @claudespice in #14208

Full Changelog: https://github.com/spiceai/spiceai/compare/v2.3.1...v2.3.2

Spice v2.3.1 is now available! 🛠️

Spice v2.3.1 is a patch release that improves performance of filtered Vortex scans, cache eviction, and query planning. It also improves installation reliability, subquery fallback, numeric conversion, and Cayenne writes.

Highlights in v2.3.1 include:

• Ordered Full Refreshes — Cayenne applies sort_columns when a full refresh replaces a table
• Faster Vortex Scans — selective filters read fewer bytes, and zones skip unrelated scan splits
• Faster IN Lists — Vortex uses a set probe for large constant lists
• Reusable Parameterized Plans — all parameter values use the same cached plan without sharing results
• Reliable Cache Eviction — declared keys and timestamp ties do not prevent eviction
• Installer and Container Reliability — the installer validates each archive, and scratch images provide a writable temporary directory
• Cayenne Memory-Mode DML — DELETE, UPDATE, and INSERT is now supported when the acceleration uses the default in-memory mode

What's New in v2.3.1​

Cayenne Applies sort_columns During Full Refreshes​

Cayenne previously applied sort_columns only during compaction. A dataset with refresh_mode: full replaces its table, so it did not run that compaction. Cayenne now sorts the replacement stream before it writes the refreshed table, and the ordered files let min/max statistics exclude unrelated files from selective queries.

In the reported TPC-H SF1 benchmark, point-lookup p50 latency improved by 16%, and the same query used 37% less CPU. The sort adds work to each full refresh, so only datasets that configure sort_columns pay this cost. See Full Refresh for full-refresh configuration.

Faster Vortex Scans​

Vortex now waits for a pushed-down filter before it prepares projected column readers, so a split with no matching rows does not prepare those readers. Selective queries in the reported benchmarks read 45% to 58% fewer bytes, and bandwidth-constrained storage showed the largest latency improvements.

Vortex also checks each scan split against its zone statistics, and it returns an empty stream when the split cannot contain a matching row. In the reported TPC-H SF1 benchmark, point-lookup p50 latency improved by 12%, and CPU use per query fell by 53%. Nonselective scans do not use these shortcuts and remain unchanged within the reported run-to-run variation.

Large IN Lists Run Faster​

Vortex previously evaluated a constant IN list once for every row and every list item, so runtime grew with both dimensions. Vortex now builds a set once and probes it in one pass, and it uses value intervals to reject files before a scan.

A 32,768-value list over 1,048,576 rows fell from 44.57 seconds to 225 milliseconds in the reported benchmark. Small lists keep their existing evaluation path.

Parameterized Queries Reuse Logical Plans​

The logical-plan cache previously included each bound parameter value in its key, so each value tuple created another entry for the same SQL text. The cache now stores an unbound plan for each SQL statement, and Spice binds the current values after it reads the plan. The SQL results cache still includes parameter values in its keys, so a result only matches a query with the same values.

The reported point-lookup benchmark improved p50 latency by 12.4% and p99 latency by 31.7%, and parameterized traffic creates fewer duplicate entries. Indexed primary-key lookups now report an exact row count to the query planner, so DataFusion can avoid an unnecessary repartition for a single-row probe.

Caching Accelerations Evict Reliably​

A refresh_mode: caching dataset can use size and row-count limits, and two defects could let the cache exceed these limits. A declared primary_key could expand the eviction key to the entire stored row, and an index marked as unique had the same fault. A Map column in the expanded key then made every sweep fail. Spice now keeps the group key limited to the request columns, and a refreshed entry also removes rows that disappeared from its latest response.

Many responses can share the same one-second fetch timestamp. The previous eviction predicate could then remove only 512 entries when this tie crossed the survivor boundary. The eviction sweep now combines a timestamp range with named boundary entries. The range removes the older bulk, so eviction converges under sustained ingestion.

Federation and Runtime Improvements​

• DuckDB and SQLite accelerations now handle more subqueries with on_zero_results: use_source. This includes mixed fallback policies and ANY or ALL comparisons.
• Decimal-to-float casts now preserve the expected rounded value after a federated aggregate. For example, a PostgreSQL average of 47.5 returns 47.5.
• Cayenne uses asynchronous filesystem calls from its asynchronous code. A slow data directory does not block the Tokio runtime at these call sites.

Installer and Container Reliability​

The runtime installer now retries unsuccessful HTTP responses and invalid archives, and it accepts an archive only when it contains a regular spiced executable. The installer sends a configured token to GitHub release requests, and it does not send that token to runtime download hosts. On native Windows, spice upgrade now rejects the unsupported runtime upgrade instead of continuing with a partial upgrade.

The scratch-based standard and release images now provide a writable temporary directory at /app/tmp, which supports operations such as acceleration snapshot uploads.

Cayenne Memory-Mode DML Works as Expected​

A Cayenne acceleration uses mode: memory by default. Before this release, filtered DML statements did not change its in-memory rows as requested.

• A filtered DELETE reported zero affected rows and left each matching row in place.
• An UPDATE could create a duplicate live row for the same primary key.
• An INSERT over an existing primary key could keep both versions of the row.

DELETE, UPDATE, and INSERT now update the in-memory tier. The append path also enforces each declared primary_key according to its on_conflict setting. The SQL DML reference lists the supported statements.

Dependency Updates​

No dependency changes its version in this release. DataFusion remains at v54.1.0, Arrow remains at v58.3.0, and Vortex remains at v0.79.0. Spice updates the Arrow and DataFusion fork revisions for the Decimal-to-Float rounding fix and the scan improvements described above.

Contributors​

• @bjchambers
• @Jeadie
• @krinart
• @lukekim
• @peasee
• @phillipleblanc

Breaking Changes​

No breaking changes.

Cookbook Updates​

No new cookbook recipes.

The Spice Cookbook includes more than 104 recipes to help you get started with Spice quickly and easily.

Upgrading​

To upgrade to v2.3.1, use one of the following methods:

CLI:

spice upgrade

Homebrew:

brew upgrade spiceai/spiceai/spice

Docker:

Pull the spiceai/spiceai:2.3.1 image:

docker pull spiceai/spiceai:2.3.1

For available tags, see DockerHub.

Helm:

helm repo update
helm upgrade spiceai spiceai/spiceai --version 2.3.1

AWS Marketplace:

Spice is available in the AWS Marketplace.

What's Changed​

Changelog​

• fix(arrow): report an exact row count from the indexed point-lookup scan by @krinart in #13972
• fix(caching): keep a declared key from disabling eviction and stranding stale rows (fixes #13976) by @bjchambers in #13992
• fix(deps): bump arrow-rs to correctly-rounded Decimal→Float cast (closes #13978) by @Jeadie in #14012
• fix(caching): partition doomed entries at the survivor cutoff so eviction converges (closes #13994) by @Jeadie in #14021
• Fix subqueries with use_source acceleration by @phillipleblanc in #14022
• fix: restore OSS installer, CLI and test workflow coverage by @phillipleblanc in #14025
• perf(vortex): defer projection setup on filtered scans until the filter resolves by @bjchambers in #14035
• fix(cayenne): make DELETE, UPDATE and INSERT work on a mode: memory acceleration (fixes #12008) by @bjchambers in #14047
• fix(deps): bump arrow-rs fork pin for Decimal->Float rounding fix by @Jeadie in #14049
• perf(vortex): answer a constant IN list by probing a set, and falsify it by interval by @bjchambers in #14061
• fix(cayenne): apply sort_columns with refresh_mode: full by @peasee in #14063
• perf(vortex): skip a scan split whose zones cannot satisfy the filter by @peasee in #14064
• perf(cache): key the logical-plan cache on SQL text, not parameter values by @bjchambers in #14069
• fix(cayenne): move accelerator filesystem I/O off Tokio workers by @lukekim in #14073
• fix: Provide temporary directory in docker images by @Jeadie in #14089

Full Changelog: https://github.com/spiceai/spiceai/compare/v2.3.0...v2.3.1

Spice v2.3.0 brings performance improvements, broader query federation, and expanded data connector capabilities. The release improves cache reuse and reduces memory use for cached SQL results. It also extends BigQuery support and adds GitHub review, release, and repository data for SQL analysis. Google models now use Vertex AI, so deployments with credentials for Google AI Studio require migration.

Highlights in v2.3.0 include:

• SQL Federation Improvements — bug fixes for function translation, NULL handling, and timezone declarations
• BigQuery Federation — temporal expressions, recursive CTEs, correlated subqueries, and window aggregates now run as one BigQuery job
• Caching Accelerator Limits — caching_max_size and caching_max_items bound what a refresh_mode: caching accelerator holds
• Stale-While-Revalidate Across a Refresh — configured stale-while-revalidate windows retain cached results across acceleration refreshes
• GitHub Review Data — eight new tables, 17 new pulls columns, and a repo / owner column on every row
• Google Models on Vertex AI — a from: google model authenticates as a GCP service account

What's New in v2.3.0​

SQL Federation Improvements​

This release improves SQL translation and function handling for accelerated and federated datasets. The following bug fixes cover string functions, NULL handling, correlated subqueries, and timezone declarations.

• A trim call failed on DuckDB, SQLite, and MySQL because DataFusion emitted its canonical name, btrim. DuckDB now receives trim, with an explicit space argument for the one-argument form. The native SQLite and MySQL paths evaluate btrim locally.
• DuckDB federation now lower-cases to_hex output to match local evaluation.
• DuckDB federation now decodes sha256 output into a 32-byte digest instead of a hexadecimal string.
• concat uses || on DuckDB to consistently propagate NULL arguments.
• inner_product now returns NULL for an undefined dot product on DuckDB for consistency.
• Operations that DuckDB cannot handle (such as regex with U or R flags) are executed locally.
• User-defined functions registered after startup now execute locally.
• Catalog connectors now use the same list of local Spice-only functions as the data connectors.
• Two EXISTS shapes emitted SQL which evaluated the correlation over the whole relation, so the bound selected nothing. A semi or mark join then reported a match on a row the plan never read. Both shapes now refuse pushdown and run locally.
• DuckDB labels a TIMESTAMPTZ column with the connection's own timezone. The connector built its pool and never pinned that setting. A dataset's schema therefore carried the host timezone, and the same query returned different rows on different machines. A connector session is now pinned to UTC.

BigQuery Federation​

BigQuery federation runs more query shapes as one remote job.

• Temporal expressions, recursive CTEs, and integer division now keep their results and stay in one federated statement. These shapes previously failed remotely, split into several queries, or produced wrong cohort boundaries.
• Three more statement shapes now run. The first is a grouped query that projects a wrapped form of its grouping expression. The second is a query whose federated tables all sit inside a correlated subquery. The third is any aggregate window function.
• A query that reads BigQuery tables from several datasets of one project now runs as one BigQuery query. It also no longer returns rows from the wrong dataset when two datasets hold a table of the same name.
• A dataset that stores JSON in a STRING column now pushes down scalar json_as_text expressions and json_get(...) IS NULL checks.
• The built-in date_trunc is preserved, and the dialect and the federation policy now agree on which aggregate and window calls are eligible.
• regexp_match null checks federate safely.
• A distinct union now renders as UNION DISTINCT. BigQuery rejects a bare UNION, so such a query failed outright. UNION ALL is unchanged.
• A numbering function such as ROW_NUMBER no longer carries a window frame, which BigQuery rejects. An aggregate window function keeps its frame.
• Percentile functions and grouping keys now render in the form BigQuery accepts. A reported 29-statement workload that failed on these shapes now runs in full.
• array_element translates a non-negative integer literal index that fits in Int64 as SAFE_ORDINAL. Other indexes evaluate locally. This preserves DataFusion's end-relative semantics for negative indexes.

Cancellation: A BigQuery query whose client goes away now stops. The BigQuery job ends as cancelled, and the pooled connection returns immediately. Before this release the query ran to completion, and the connection stayed busy for its whole duration. Enough cancellations left an application unable to query at all.

The Caching Accelerator Accepts Explicit Limits​

A refresh_mode: caching accelerator had nothing bounding what it held. Retention was derived from caching_ttl plus caching_stale_while_revalidate_ttl, and only when caching_stale_if_error was disabled. A dataset that set caching_stale_if_error therefore got no policy at all, and nothing was ever evicted. Nothing capped the acceleration by size or by count either.

Two settings now bound the acceleration. Each one refuses an unparseable value rather than falling back to a default:

• caching_max_size — a byte budget, such as 512MiB.
• caching_max_items — a row budget.

caching_ttl is also accepted as caching_item_ttl, which is the spelling the SQL results, search results, and embeddings caches use. Eviction is entry-granular. A cached response can span several rows, so the runtime ranks entries by their oldest page and removes all of an entry's rows together. The storage schema is unchanged, and no existing acceleration needs a rebuild.

datasets:
  - from: https://api.example.com/v1/items
    name: items
    acceleration:
      enabled: true
      engine: duckdb
      refresh_mode: caching
      primary_key: '(request_query, request_path)'
      params:
        caching_ttl: 5m
        caching_max_size: 512MiB
        caching_max_items: 50000

caching_stale_if_error now fires on the failure it exists for. It keyed off a fetch that returned an error. The HTTP connector reports a failing origin as a successful fetch whose rows carry a 429 or 5xx status once it exhausts max_retries. An operator who enabled the setting received the origin's error body instead of the cached response.

A caching accelerator that has nothing bounding it now says so at startup.

SQL Results Cache Improvements​

Stale results remain available across a refresh when configured. An acceleration refresh evicted every dependent SQL results cache entry, and any successful refresh counted as a change. A refresh_mode: full update still flushed the whole per-table cache. For a workload with consistently high QPS, each refresh turned a population of cached results into simultaneous synchronous misses.

When stale_while_revalidate_ttl is configured, an invalidation now marks dependent entries stale as of the refresh instead of evicting them. Inside the stale window the runtime serves the previous result with Results-Cache-Status: STALE and starts one background revalidation per key. Past the stale window the request is a miss, exactly as before. With no stale window configured, invalidation stays hard.

Cache accounting covers more retained memory. A cache with a million empty results reported 0.09 GiB against 1.85 GiB of retained memory. Its max_size accounting omitted parts of each entry. The cache now shares schemas and input-table sets, copies foreign buffers, and accounts for per-buffer allocation overhead.

In the reported benchmark, each entry retained 1%–80% less memory across 20 combinations of result shape and source. The benchmark ran on macOS/arm64 with snmalloc. Reported size ranged from 0.65x to 1.73x of retained memory after the change. The lowest ratio before the change was 0.31x. These figures compare the fix with its merge base, not v2.2.1, and are not guarantees for every workload.

This release also corrects memory accounting for the search results and embeddings caches.

Pingora cache engine: Table invalidation read every entry with a destructive get, so an invalidation promoted every key in the cache. Scan order replaced recency, and each visited key became a momentary miss to concurrent readers. A read also served a hit destructively, so a second reader reported a miss for a key the cache holds. Both reads are now non-destructive.

GitHub Data Connector: Review, Release, and Repository Tables​

The GitHub Data Connector adds eight tables, 17 columns on pulls, and repository identity on every row. An application can now answer a code-review question in SQL. Before this release, pulls.reviews_count was a bare integer with no state and no reviewer, and pulls.review_comments recorded only inline comments.

pulls adds is_draft, mergeable, merge_state_status, review_decision, status_check_rollup, merge_queue_state, merge_queue_position, merged_by, closed_by, base_ref, head_ref, head_sha, milestone_id, milestone_title, closing_issues_references, closing_issues_count, and reactions_count. issues adds state_reason, closed_by, reactions_count, type, and type_color. Every table returns a repo and an owner column, so a multi-repository UNION ALL keeps its rows apart.

The connector also asks GitHub for a narrower pull request page. A 100-node page now exceeds GitHub's per-request compute budget on a large repository. GitHub rejects that page with Resource limits for this query exceeded and returns every node as null, so the dataset never loaded.

Google Models Move to Vertex AI​

A from: google chat or embedding model now authenticates as a GCP service account against Vertex AI. Spice no longer accepts a Google AI Studio API key. See Breaking Changes for the migration.

models:
  - from: google:gemini-2.5-pro
    name: gemini
    params:
      google_project: my-project
      google_location: us-central1
      google_service_account_path: /etc/spice/gcp-sa.json

Other AI model fixes in this release:

• An Anthropic model configured without an explicit model id now resolves. The default named claude-3-5-sonnet-latest, which Anthropic has retired, so every such request failed.
• An Anthropic model now refuses an OpenAI top_logprobs request instead of translating it to top_k. The two fields are unrelated. top_logprobs reports log probabilities and top_k narrows sampling, so the translation silently changed the model's output.
• Anthropic streaming failures and provider refusals from openai, xai, and spiceai are now classified from the provider's typed error fields. Each path searched the rendered error text for 401, 429, or rate, and then replaced the provider's own detail with a fixed string.
• A from: huggingface: chat model reads hf_token again. The parameter moved to the prefix huggingface, so hf_token was warned about as unknown and a gated repository was downloaded anonymously.
• An Amazon Bedrock model now names the credential AWS rejected instead of reporting unhandled error.
• The text-embeddings-inference model-load path runs its filesystem and tokenizer work on a blocking thread. That work ran on a Tokio worker thread during model registration, so it could starve /health.

Search Improvements and Bug Fixes​

This release fixes bugs in search result limits, index updates, and deletes.

• A result set larger than the requested limit: vector_search(tbl, 'query', 10) against an Elasticsearch-backed index now respects the requested limit.
• Index writes and deletes kept in sync with the table: Previously, writes for rows with repeated primary keys, no chunks, or non-embeddable chunks could leave the previous index entry. These outdated index entries are now deleted.
• A chunked Elasticsearch delete: The delete filtered on the key columns, and a string key was left to Elasticsearch dynamic mapping as an analyzed text field. The delete now filters on a field that can match the key exactly.
• A partial Elasticsearch delete: _delete_by_query returns 2xx when the request ran, and it reports per-document failures and version conflicts in the body. Spice discarded that body, so a delete could leave documents behind and still report success.
• Full-text index encoding: A full-text upsert is a delete followed by an insert, so both halves must encode the primary key the same way. They disagreed for Float32, Float16, and Binary keys, and both the old and the new row stayed in the index.

Acceleration and Refresh​

• LIMIT on a partitioned scan: PartitionTableProvider::scan passed the scan limit as the skip argument rather than the fetch argument. LIMIT 10 over a three-row partitioned dataset returned zero rows.
• acceleration.enabled: false: A dataset or a view can set enabled: false and leave the rest of the block in place. The runtime read every other setting, accepted it, and then ignored it. The component reported healthy and served federated queries. The runtime now names the settings it discards.
• ready_state on a view: A view's acceleration.ready_state was accepted by the schema and by the parser, and then never applied. The identical key on a dataset was honoured. A view now resolves the key the way a dataset does.
• A retention policy that cannot start: A dataset that set retention_check_enabled: true, a retention_period, and a time_column but no retention_check_interval got no retention task and no diagnostic. The builder now reports the refusal.
• A refresh completion that arrives early: A completion published before a caller registered its wait was dropped with no record, and the caller waited for a refresh that had already happened. The signal is now level-triggered.
• A refresh completion from the wrong refresh: A waiter was satisfied by the next completion recorded on the table, whichever refresh produced it. A refresh that was already running could therefore release a caller. Completions are now correlated with the refresh that a caller triggered.
• A table replaced during a refresh: Two callers acted on a completion for a table that had since been removed or rebuilt. The runtime now re-resolves the table after the refresh lands and before it acts on the completion.
• A schema repair on a checkpoint: Writing a checkpoint's schema also wrote its refresh timestamp, so a schema repair told the scheduler the data was fresh. An overdue dataset then waited a full refresh_check_interval. A schema repair now leaves the freshness clock alone.
• A recorded snapshot schema: A snapshot's recorded schema is a foreign declaration, and a Map that declares its entries field nullable is a declaration no accelerator can hold. The restore path now conforms that declaration to the Arrow map layout.
• A source row durable write-back could not confirm: The delivery worker read a missing point-scan row as a deletion. A short visibility gap in the accelerator therefore deleted that row from the source of record. The worker now withholds a key it cannot read and retries it on a later pass. A delivery cursor advances only after the pass succeeds. Write-back also refuses a configuration it cannot uphold, which is a behavior change. See Breaking Changes.

Arrow and Storage Bug Fixes​

• Decimal128 on write paths: Three conversions could produce a plausible wrong number instead of an error. The sum of two in-range halves wrapped to a large negative decimal at scale 38. A float-to-int cast saturated to i128::MAX, and NaN became 0. The declared precision was never checked, so a value needing more digits than the column declares was appended anyway. All three were reachable from Debezium decimal ingestion, where the input is source-controlled. The runtime now validates the destination precision once, where every input form converges.

• An Iceberg DELETE an equality key cannot express: An Iceberg delete writes an equality delete file, which removes rows whose key columns equal the given values. That statement matches the user's WHERE only when the condition reads key columns alone. A condition on a float or a nested column removed rows that did not match. Spice now refuses the statement and names the offending column.

• A Parquet object overwritten mid-scan: A listing-table Parquet scan decoded two object generations as one file. The scan now pins one generation through a version id or an If-Match header. dataset_acceleration_refresh_errors carries reason=object_generation_changed|parquet_decode|other, so an expected overwrite is distinguishable from corruption.

• An Arrow relabel: relabel_array_data carries an array's values across a type change untouched, and only field names and nested nullability flags may differ. Nothing enforced that contract. It now refuses three kinds of target:

  • a target that changes what the buffers mean
  • a target that declares away nulls the array still holds
  • a target whose same-typed sibling fields are reordered

  The third kind produced silent column-value transposition on the Delta Lake column-mapping path.

• Arrow MAP columns: The Arrow map layout forbids a nullable entries field, and MapArray::try_new refuses one. Nothing enforced it at decode. A producer that declared it that way handed over a column that decoded cleanly and then failed in the first kernel that rebuilt it. The runtime now normalizes entries nullability at every Arrow decode point. A Databricks SQL Warehouse MAP column, which declares entries nullable, no longer panics the runtime. A Cayenne accelerator that has already persisted the non-conforming declaration is now repaired.

• A nested nullability difference: try_cast_to decided its fast paths with Schema::contains, which permits a nested field's nullability to differ. RecordBatch requires the two types to be identical. The shared entry point now aligns the difference instead of publishing it.

• A retired Vortex file: Retiring a Cayenne file released its Vortex segments and left its footer in DataFusion's file-metadata cache, which has no TTL. A file opened during retirement could also repopulate the path that retirement had just cleared. The retirement drain had no ceiling. A stalled put therefore held every caller of the invalidation, and the delete sink is one of them. All three faults are fixed.

• A Cayenne teardown that deleted a shared metastore: Recreation of a Cayenne dataset could delete a shared metastore and leave other datasets unavailable after restart. The guard checked only the metastore named by that dataset's settings, not catalogs inside its data directory. Open file handles hid the loss until restart. The runtime now scans that directory before catalog changes and again before deletion. It refuses recreation if it finds a Cayenne metastore or cannot safely resolve the configured paths.

Observability and Operations​

• OpenTelemetry resource attributes: The OTLP ingest path parsed resource attributes such as service.name and service.instance.id and then dropped them. Data points from two processes were therefore indistinguishable once written. Resource attributes now reach the metric data points. The same change closes four ingest races that dropped data with no error. One of those races let a write publish through a table provider that a schema evolution had already replaced.
• A panicking query: A query whose execution panicked was sometimes returned as an empty HTTP 200 success, which no client can tell apart from "no rows matched". This happened in 20 of 60 identical runs on trunk. A panicking query is now always an error.
• runtime.cpu.cores above the container's ceiling: runtime.cpu.cores was the one CPU entitlement setting taken raw rather than clamped. A pod configured with runtime.cpu.cores: 6 under resources.limits.cpu: 2 sized every derived pool for six cores and was then throttled. The runtime now warns and names both readings. It does not clamp, because an operator may size the runtime for a node the pod has not reached yet.
• HTTP latency: The HTTP server sets TCP_NODELAY, which lowers the latency of a small response body. The Flight SQL server already set it.
• MCP tools in runtime.task_history: A proxied MCP tool call was recorded under two different task values depending on the entry point, so one logical tool split across two rows. Grouping by task gave wrong per-tool counts. Both entry points now use the encoded name.
• A discarded Flight batch: The runtime reported a data_loss count that counted a message by its body length. A batch whose body is empty still carries rows, so the count was wrong. The runtime now reads the IPC header.
• A hot reload that changes functions or catalogs: A cached logical plan embeds the ScalarUDF and the TableSource it was planned against. A hot reload that redefined a SQL function or replaced a catalog left those plans in place. The same SQL then kept answering from the replaced component. The plan cache is installed unconditionally with a one-hour TTL, so no caching configuration was needed to hit this. Both handlers now discard the affected plans.
• Cloud Connect metrics cadence: A Cloud Connect instance exports metrics every 10 seconds rather than every 30, so a chart drawn from the control stream resolves at 10 seconds. The payload is a snapshot of cumulative totals, so this changes chart resolution and not what is recorded.

Other Improvements and Bug Fixes​

• Glue catalog: A discovered Glue table that Spice cannot read, such as an ORC or Avro table, was absent from the catalog with nothing said about it. The catalog connector now reports each such table and the reason.
• Databricks: The connector accepts Unity Catalog streaming tables and views. It also forwards the runtime's spark feature.
• DuckDB index materialization: The DuckDB intermediate index materialization rule reads a table's index list before it rewrites a scan into a materialized CTE. That list had been empty, so the rule never fired and an indexed column never narrowed a scan.
• Vortex list_length pushdown: DataFusion array_length(expr) and array_length(expr, 1) now convert to Vortex list_length and push into the scan. A list length is computed from offsets, and element values are not materialized.
• ScyllaDB: The ScyllaDB Data Connector is out of the default build, alongside ODBC. make install-scylladb or --features scylladb builds it. A Spicepod that names scylladb: on a build without it now says the build lacks the connector rather than offering the closest registered name. The connector also declines a physical sort that CQL cannot serve.
• Turso: The accelerator refuses a stored list whose encoding predates the version marker rather than reading it under the current encoding.
• CLI: spice run resolves spiced beside the CLI before it reaches for the managed install, and never from PATH. A Spice Cloud project listing is attributed to the organization it was requested for. spice query and spice nsql analyze keep the API key on its origin across a redirect. A Cloud Connect managed instance no longer warns about the default pods watcher on every spice run.
• MCP tools: A renamed tool forwards strict() and as_mcp_proxy() to the tool it wraps. runtime-tools declares what its mcp feature needs.
• Connector registries on shutdown: The runtime no longer clears stateless connector registries on shutdown.

Dependency Updates​

Compared with v2.2.1, this release changes the following versions:

DataFusion remains at v54.1.0 and Arrow remains at v58.3.0. Spice updates their fork revisions for the federation, Parquet scan, and cache fixes described above.

Contributors​

• @bjchambers
• @Jeadie
• @krinart
• @lukekim
• @peasee
• @phillipleblanc
• @sgrebnov
• @vatsalp2008

Breaking Changes​

Google models authenticate against Vertex AI. A from: google chat or embedding model no longer accepts google_api_key. Every such model now authenticates as a GCP service account.

Update each from: google model and embedding to set google_project, google_location, and exactly one credential setting.

Before:

models:
  - from: google:gemini-2.5-pro
    name: gemini
    params:
      google_api_key: ${secrets:google_api_key}

After:

models:
  - from: google:gemini-2.5-pro
    name: gemini
    params:
      google_project: my-project
      google_location: us-central1
      google_service_account_path: /etc/spice/gcp-sa.json

Set exactly one of the three credential settings.

Two behavior changes to note before you upgrade:

• Durable write-back rejects unsafe settings and operations. Before you upgrade, set mode: file, remove retention settings, and declare a single-column primary_key for each durable write-back dataset. The runtime rejects unsupported settings at load time. Submit writes inside a transaction as one BEGIN; ...; COMMIT; request. Write-back datasets reject DELETE and TRUNCATE when you issue those statements.
• The ScyllaDB Data Connector is out of the default build. Build with --features scylladb, or run make install-scylladb, to keep it. ODBC already worked this way.

Cookbook Updates​

No new cookbook recipes.

The Spice Cookbook includes more than 104 recipes to help you get started with Spice quickly and easily.

Upgrading​

To upgrade to v2.3.0, use one of the following methods:

CLI:

spice upgrade

Homebrew:

brew upgrade spiceai/spiceai/spice

Docker:

Pull the spiceai/spiceai:2.3.0 image:

docker pull spiceai/spiceai:2.3.0

For available tags, see DockerHub.

Helm:

helm repo update
helm upgrade spiceai spiceai/spiceai --version 2.3.0

AWS Marketplace:

Spice is available in the AWS Marketplace.

What's Changed​

Changelog​

• fix(search): surface an Elasticsearch delete that only partially applied (fixes #12364) by @claudespice in #12720
• fix(ci): bound every integration job, so a wedged one cannot hold the queue (refs #12718) by @grokspice in #12721
• feat(tools): add pdf-parse tool to compare liteparse and pdf-inspector by @Jeadie in #12807
• fix(turso): refuse a stored list whose encoding predates the version marker (fixes #12632) by @grokspice in #12837
• Use TypedParams for reranker parameters by @Jeadie in #13046
• fix(search): correctness, async-safety, and performance fixes across the search subsystem by @Jeadie in #13065
• Decide full-text CDC-attachment at construction, not after by @Jeadie in #13075
• fix(scylladb): decline the physical sort pushdown CQL cannot serve (fixes #10775) by @claudespice in #13107
• fix(cache): stop Pingora table invalidation from promoting every key it reads (fixes #12674) by @claudespice in #13117
• fix(cayenne): stop a failed statistics read publishing a partial count as exact (fixes #13010) by @claudespice in #13125
• fix(cayenne): refuse a widened CDC batch a partitioned acceleration cannot apply (fixes #13051) by @claudespice in #13133
• fix(catalogs): report the Glue tables Spice cannot read instead of dropping them silently (fixes #13102) by @claudespice in #13149
• fix(ci): gate every tracked Rust source tree in the merge queue's change filter (fixes #13120) by @claudespice in #13151
• fix(cache): bound max_size on the memory an entry holds, not its array bytes (fixes #12931) by @claudespice in #13154
• fix(cayenne): honour the sort-merge row floor on the memory-gated path (fixes #12958) by @claudespice in #13157
• fix(cache): serve a Pingora hit without hiding it from a concurrent reader (fixes #12987) by @claudespice in #13158
• fix(vortex): stop a file opened mid-retirement from repopulating the cleared path (fixes #12963) by @claudespice in #13161
• fix(vortex): evict a retired file's footer with its segments (fixes #12953) by @claudespice in #13162
• feat(google): switch from Google AI Studio to Vertex AI by @krinart in #13210
• Run TEI candle model-load filesystem and tokenizer work on a blocking thread by @Jeadie in #13223
• fix(correctness): validate Decimal128 precision and overflow on write paths by @lukekim in #13224
• fix(runtime): stop clearing stateless connector registries on shutdown by @Jeadie in #13225
• fix(search): use one encoding path for full-text index writes and deletes by @Jeadie in #13226
• fix(docker): install ARM64 linker toolchain by @Jeadie in #13266
• ci: build the runtime integration archives with one feature set by @bjchambers in #13270
• ci: archive every integration test target in one invocation by @bjchambers in #13273
• fix(sql): bump the datafusion pin so a bounded EXISTS refuses instead of returning wrong rows (fixes #13277) by @claudespice in #13280
• ci: have the compiler-cache action own its credentials by @bjchambers in #13282
• fix(cpu-budget): warn when a configured cores value exceeds the container's real CPU ceiling (fixes #13275) by @claudespice in #13284
• fix(databricks): publish a MAP column with the non-nullable entries field Arrow requires (fixes #7307) by @claudespice in #13288
• fix(search): bound a search result set by the requested limit, not just the index read (fixes #13274) by @claudespice in #13290
• fix(vortex): bound the segment-cache retirement drain so a stuck put cannot hold a delete open (fixes #12964) by @claudespice in #13300
• Reach the accelerator contract without going through the runtime by @bjchambers in #13304
• fix(postgres): rebuild the datasets already streaming when a join replaces their replication slot (fixes #13229) by @claudespice in #13313
• fix(cli): resolve spiced beside the CLI before the managed install, and never from PATH by @claudespice in #13317
• fix(iceberg): refuse a delete an equality key cannot express by @lukekim in #13322
• Stop the accelerator engines reaching up into the runtime by @bjchambers in #13324
• fix(benchmarks): open snapshot-update PRs against the dispatch branch, not always trunk by @krinart in #13335
• chore(deps): bump datafusion-table-providers for NUMERIC scale fidelity by @lukekim in #13349
• feat(cloud-connect): export metrics every 10s by @phillipleblanc in #13353
• Move the accelerator engines into their own crates by @bjchambers in #13354
• perf(cayenne): shard the encode by what the write is, not by what the table declares by @lukekim in #13356
• fix(cli): attribute a project listing to the org it was requested for by @lukekim in #13357
• Enable Oracle TPC-H result validation, fix the loader that trimmed leading spaces by @sgrebnov in #13360
• Readme: Change PG catalog status from Alpha to Beta by @sgrebnov in #13361
• Upgrade iceberg-rust to v0.10.1 by @krinart in #13365
• perf(cayenne): cut a rewrite's shards at equal row mass, not equal width by @lukekim in #13373
• fix(deps): bump datafusion-table-providers for AVG/division NUMERIC scale rounding by @krinart in #13387
• fix(databricks): forward runtime's spark feature to the databricks connector by @sgrebnov in #13389
• Move the Cayenne accelerator into its own crate by @bjchambers in #13391
• fix(testoperator): compare exact decimals on their mantissas, not through f64 by @bjchambers in #13407
• Configure an accelerator engine through its constructor instead of a published global by @bjchambers in #13409
• fix(opentelemetry): merge resource attributes into metric data points and close OTLP-to-sink ingest races by @peasee in #13412
• fix(arrow): refuse a relabel that changes what the array's buffers mean (fixes #13423) by @claudespice in #13435
• fix(tools): record one task_history task per proxied MCP tool (fixes #13338) by @claudespice in #13437
• fix(release): read release notes from a file and trim them to GitHub's body limit by @sgrebnov in #13457
• fix(ci): probe the cc toolchain before reaching for brew in setup-cc by @claudespice in #13466
• fix(cayenne): refuse a teardown that would delete a metastore no parameter names (fixes #13436) by @claudespice in #13471
• fix(spiced): stop warning about the default pods watcher by @phillipleblanc in #13494
• fix(arrow): align a nested nullability difference instead of advertising it (fixes #13285) by @claudespice in #13496
• fix(acceleration): make the refresh-completion signal level-triggered (fixes #13086) by @claudespice in #13505
• fix(acceleration): apply a partitioned scan's LIMIT as a fetch, not a skip by @vatsalp2008 in #13507
• fix(ci): stage the retention OOM test binary without its debug info by @claudespice in #13511
• fix(ci): let sign-off run from a worktree nested inside the checkout, and add --skip-targeted by @bjchambers in #13520
• Authenticate the stargazers dataset with a PAT and drop qa_analytics by @lukekim in #13529
• fix(ci): isolate sccache per job on shared self-hosted Macs by @lukekim in #13531
• chore(scylladb): take the ScyllaDB connector out of the default build by @lukekim in #13532
• feat(github): add review, release, milestone, user and repo tables, plus repo/owner columns by @lukekim in #13545
• fix(ci): match the allowed refresh-task warning at its current module path by @claudespice in #13547
• fix(flight): normalize MAP entries nullability at every Arrow decode point (fixes #13495) by @claudespice in #13550
• fix(anthropic): default to a model Anthropic still serves (fixes #13557) by @claudespice in #13563
• fix(postgres): rebuild an emptied CDC acceleration rather than resume its surviving position (refs #13546) by @claudespice in #13566
• chore(spicepod): accelerate GitHub datasets with Cayenne instead of DuckDB by @lukekim in #13571
• fix(arrow): refuse a relabel that declares away nulls the array still holds (fixes #13433) by @claudespice in #13585
• fix(cayenne): refuse a catalog whose data directory would hold its metastore (fixes #13105) by @claudespice in #13593
• fix(ci): exempt every App account from the assignee gate, not just Dependabot (fixes #13115) by @grokspice in #13594
• fix(mysql): assign the binlog dump session's net_write_timeout floor as an integer literal (fixes #13307) by @grokspice in #13595
• fix(ci): let the E2E macOS build share the fleet's Cargo home (fixes #13299) by @grokspice in #13596
• fix(ci): call a test binary the runner cannot load an infrastructure failure (fixes #13518) by @grokspice in #13597
• fix(spicepod): say which acceleration settings enabled: false discards (fixes #13514) by @grokspice in #13602
• feat(caching): bound a caching accelerator by size, count and entry lifetime (closes #13525) by @bjchambers in #13604
• fix(ci): catch a stale Cargo.lock before the merge queue, not after a 55-minute build (fixes #13598) by @grokspice in #13606
• fix(bedrock): say which credential AWS rejected instead of "unhandled error" (refs #12396) by @claudespice in #13616
• feat(caching): serve results stale after an acceleration refresh instead of evicting them by @krinart in #13618
• fix(write-back): never delete a source row for a key the accelerator did not return by @phillipleblanc in #13638
• fix(cayenne): fold a staged append's unpublished keys into the PK-keyset rebuild (fixes #13639) by @claudespice in #13644
• fix(tools): forward strict() and as_mcp_proxy() from a renamed tool (fixes #13443) by @claudespice in #13649
• fix(delta_lake): order a column-mapping relabel target the way the scan reads it (fixes #13434) by @claudespice in #13655
• fix(ci): raise the e2e Linux build bound above the worst legitimate run (fixes #13674) by @grokspice in #13675
• fix: comment out refresh_append_overlap in the sample spicepod by @lukekim in #13680
• fix(anthropic): refuse a log-probability request instead of narrowing sampling (fixes #13581) by @claudespice in #13682
• fix(cayenne): record a pipelined non-conflict staged append's primary keys (fixes #13642) by @claudespice in #13686
• fix(schema): stop an illegal Arrow Map entries declaration from being stored or compared (fixes #13549) by @claudespice in #13695
• build(rust): upgrade toolchain to 1.97.1 by @lukekim in #13696
• fix(ci): stop a CI git push from blocking forever on a credential prompt (fixes #13701) by @claudespice in #13702
• fix(search): remove the vector a rejected write left behind (fixes #13504) by @claudespice in #13705
• fix(acceleration): stop a refresh already running from answering a later waiter (refs #13544) by @claudespice in #13709
• fix(search): remove a chunked row's stale chunks when its text goes away (refs #13704) by @claudespice in #13716
• fix(catalogs): install the Spice function deny-list on the SQL catalog connectors (refs #13664) by @claudespice in #13731
• fix(runtime-tools): declare what the mcp feature actually needs (fixes #13648) by @grokspice in #13733
• fix(runtime): re-resolve a table after its refresh lands, before acting on the completion (fixes #13603) by @claudespice in #13735
• fix(flight): count a discarded batch by its IPC header, not its body length (fixes #13636) by @grokspice in #13736
• docs(makefile): say what SPICED_DATA_FEATURES actually is (fixes #13678) by @grokspice in #13738
• fix(anthropic): classify a streaming failure by Anthropic's error type, not its message text (fixes #13562) by @claudespice in #13748
• fix(views): apply a view's acceleration.ready_state instead of dropping it (fixes #13615) by @claudespice in #13750
• fix(ci): resolve a Python 3.11+ interpreter for the lint-rust guards (refs #13754) by @grokspice in #13755
• fix(bigquery): emit valid pushed-down SQL by @phillipleblanc in #13768
• fix(bigquery): safely federate regexp_match null checks by @krinart in #13771
• feat(hash-index): verify the bloom filter's block index with Verus by @lukekim in #13777
• fix(adbc): federate a BigQuery statement spanning datasets as one query by @phillipleblanc in #13780
• fix(adbc): cancel an abandoned query, stop the BigQuery job, free the connection by @phillipleblanc in #13782
• fix(snapshot): conform a recorded snapshot schema to the Arrow map layout (fixes #13694) by @claudespice in #13786
• fix(cayenne): materialize the in-memory CDC tier before a scanning DELETE by @lukekim in #13798
• fix(caching): say when a caching accelerator has nothing bounding it (fixes #13525) by @claudespice in #13805
• fix(bigquery): run three federated statement shapes BigQuery was refusing by @phillipleblanc in #13812
• fix(duckdb): rewrite DataFusion's btrim to DuckDB's trim (fixes #13794) by @claudespice in #13821
• fix(federation): stop pushing btrim to SQLite and MySQL, which have no btrim (fixes #13840) by @claudespice in #13823
• fix(ci): give the throughput workflow the postgres fixtures bench provisions by @krinart in #13830
• fix(acceleration,cayenne): Resolve quoted columns in keys, name which primary key columns are null by @peasee in #13845
• fix: pin listing-table Parquet reads to one object generation by @phillipleblanc in #13847
• fix(duckdb): lower-case the hex digits a federated to_hex gets back (fixes #13818) by @claudespice in #13852
• fix(bigquery): carry the merged unparser fixes through the dialect wrapper by @phillipleblanc in #13853
• fix(databricks): allow Unity Catalog streaming tables and views through the table-type check by @krinart in #13855
• fix(acceleration): report a retention policy that cannot start instead of silently building none (fixes #13804) by @claudespice in #13857
• fix(search): evict a key whose deciding row the index rejected (fixes #13848) by @claudespice in #13859
• fix(duckdb): Restore intermediate index materialization optimization by @sgrebnov in #13864
• fix(federation): refuse a user function the deny-list snapshot was built before (fixes #13726) by @claudespice in #13868
• fix(duckdb): decode a federated sha256 back to the digest's bytes (fixes #13850) by @claudespice in #13869
• fix(runtime): Add TCP_NODELAY to HTTP server by @peasee in #13874
• fix(query): surface a panicking query as an error, never an empty success (fixes #13876) by @claudespice in #13878
• fix(bigquery): keep the built-in date_trunc, forward two dialect renderings, repin the unparser by @phillipleblanc in #13882
• fix(llms): classify a provider refusal from its typed fields, not its message (refs #13747) by @claudespice in #13884
• fix(vortex): keep control-byte field names unescaped in physical schema by @lukekim in #13886
• perf(vortex): push DataFusion array_length down as Vortex list_length by @lukekim in #13888
• fix(duckdb): render a federated concat as || so a NULL argument propagates (fixes #13849) by @claudespice in #13889
• fix(acceleration): let a schema repair correct a checkpoint without resetting the freshness clock (fixes #13817) by @claudespice in #13894
• fix(duckdb): screen a federated inner_product so a non-finite result is NULL (fixes #13787) by @claudespice in #13895
• refactor(postgres): report an acceleration re-read as a refresh, not a bespoke metric by @bjchambers in #13896
• fix(search): classify a partially non-finite embedding as unindexable on every backend (fixes #13872) by @claudespice in #13902
• fix(duckdb): pin a connector's DuckDB session to UTC so a dataset's schema does not carry the host timezone (fixes #13899) by @claudespice in #13903
• fix(cayenne): keep a file's statistics the same whichever source serves them (refs #13829) by @claudespice in #13904
• fix(bigquery): preserve results and federation for temporal and recursive queries by @bjchambers in #13905
• Make the SQL results cache hold what it says it holds by @bjchambers in #13908
• fix(duckdb): keep a call the dialect cannot render out of the federated plan (fixes #13900) by @claudespice in #13909
• fix(functions): discard cached plans when a hot reload changes the function set (refs #13873) by @claudespice in #13911
• fix(runtime): discard cached logical plans when a hot reload replaces a catalog (fixes #13910) by @claudespice in #13914
• fix(cli): keep the API key on its origin in the SDK-built query client (fixes #12502) by @grokspice in #13923
• fix(search): filter a chunked Elasticsearch delete on a field that can match the key (fixes #13714) by @claudespice in #13926
• ci: align default and ODBC build features by @phillipleblanc in #13933
• fix(github): bound the pull request page to GitHub's per-request compute budget (refs #13762) by @grokspice in #13938
• fix: stabilize GitHub tests and bound GraphQL registration (fixes #13762) by @lukekim in #13939
• fix(ci): preserve Cargo discovery markers during runner disk sweeps by @phillipleblanc in #13940
• fix(bigquery): push down JSON scalar text and null checks by @phillipleblanc in #13944
• fix(models): read the HuggingFace chat token as hf_token again (fixes #13932) by @claudespice in #13946
• fix(postgres): decode versioned JSONB binary replication values by @phillipleblanc in #13962
• fix(postgres): preserve microseconds in timestamp writeback by @phillipleblanc in #13963
• ci: upgrade spiceio setup action to v0.9.0 by @lukekim in #13971

Full Changelog: https://github.com/spiceai/spiceai/compare/v2.2.1...v2.3.0

Spice v2.1.5 is now available! 🛠️

Spice v2.1.5 is a patch release focused on dependable cached data and predictable operation under load. Cached queries now reflect data changes more reliably, Cayenne workloads stay within configured memory limits, health checks remain responsive while cached results are updated, and cache dashboards provide a more complete view of activity.

What's New in v2.1.5​

Cached Queries Stay Fresh as Data Changes​

Cached results are now cleared reliably after refreshes, writes, retention changes, and updates to dependent local datasets. Expired entries are removed promptly, and entries for data that is no longer part of a Cayenne dataset are not reused by later queries.

These improvements prevent a completed data change from being followed by an older cached answer. No configuration changes are required.

More Predictable Cayenne Behavior Under Heavy Load​

Cayenne now keeps track of the memory needed to prepare query results, including when several parts of a query are prepared at once. Multiple accelerated tables also share available memory instead of each planning as though it were the only table in the deployment.

Large and concurrent workloads are therefore less likely to exhaust the available memory. When a query cannot fit within the configured limit, it fails cleanly instead of putting the entire service at risk. Operators can also limit how much work one dataset does at once when it needs a smaller memory footprint, without slowing every query.

Health Checks Remain Responsive During Cache Updates​

Refreshing or writing a dataset with many cached results no longer holds up Spice while it finds old answers that need to be cleared. Health checks and other requests can continue during this work, reducing avoidable service restarts under load.

More Trustworthy Cache Dashboards​

Cache dashboards now show total space, space in use, stored results, requests, and successful reuse whenever the dashboard is refreshed, including for new or lightly used datasets. Counts for expired results, automatic size cleanup, and cleanup after data changes are also reported consistently, so a zero value represents no activity rather than missing information.

Contributors​

• @bjchambers
• @krinart
• @lukekim
• @phillipleblanc

Breaking Changes​

No breaking changes.

Cookbook Updates​

No new cookbook recipes.

The Spice Cookbook includes more than 100 recipes to help you get started with Spice quickly and easily.

Upgrading​

To upgrade to v2.1.5, use one of the following methods:

CLI:

spice upgrade

Homebrew:

brew upgrade spiceai/spiceai/spice

Docker:

Pull the spiceai/spiceai:2.1.5 image:

docker pull spiceai/spiceai:2.1.5

For available tags, see DockerHub.

Helm:

helm repo update
helm upgrade spiceai spiceai/spiceai --version 2.1.5

AWS Marketplace:

Spice is available in the AWS Marketplace.

What's Changed​

Changelog​

• fix: arrow IndexedMemTable serves stale results after DML/retention/sync (fixes #11262) by @claudespice in #11532
• fix(cache): evict the Pingora cache down to max_size instead of only recording it (fixes #12688) by @grokspice in #12694
• fix(cache): count an invalidation as an eviction, and export the counters before one fires (fixes #12687) by @grokspice in #12791
• fix(cache): never serve a result whose tables changed after it read them by @bjchambers in #12703
• Remove unnecessary allocations from results-cache and hot conversion paths by @phillipleblanc in #11895
• fix(cache): read and expire a Pingora entry under one hold of its shard (fixes #12832) by @grokspice in #12839
• fix(runtime): invalidate localpod children's cached results on parent refresh by @krinart in #12897
• fix(cache): run the Pingora invalidation scan off the calling runtime worker (fixes #12806) by @grokspice in #12808
• fix(cache): count the removals the Pingora engine performs itself (fixes #12792) by @grokspice in #12830
• feat(cayenne): charge scan materialization to the query memory pool by @lukekim in #12759
• feat(cayenne): bound the SUM of the per-table PK keyset caches by @lukekim in #12802
• Charge the query memory pool for concurrent Vortex split decodes by @lukekim in #12940
• fix(vortex): invalidate retired segment cache entries by @phillipleblanc in #12943
• fix(vortex): expose segment cache metrics on scrape by @phillipleblanc in #12944

*Full Changelog: https://github.com/spiceai/spiceai/compare/v2.1.4...v2.1.5

Spice v2.1.1 is now available! 🛠️

Spice v2.1.1 is a patch release focused on reliability and performance. It resolves a possible deadlock that affected datasets with partitioned Cayenne accelerators, caches empty SQL result sets so repeat queries are served from cache, speeds up repeated queries on multi-file datasets, and restores Bedrock embedding provider parameters.

What's New in v2.1.1​

Cayenne Partitioned Dataset Deadlock Fix​

Cayenne datasets configured with partition_by could deadlock during their initial refresh and never become ready. Non-partitioned tables and small partitioned tables were unaffected. The root cause was a deadlock between the partition routing and the global Vortex encode budget introduced in v2.1.0.

Cayenne Zero-Row Append Refresh Stability​

An idle append refresh, one where no source rows are newer than the current max(time_column), wrote no Vortex files, so the expected snapshot directory was never created. The subsequent fsync on that directory failed with ENOENT, marking the dataset unhealthy. The fix skips the snapshot sequence record and protected-snapshot publish when the write carried no rows.

Caching of Empty Result Sets​

The SQL results cache now stores empty (zero-row) result sets. Queries that legitimately return no rows (e.g. WHERE 1=0, LIMIT 0) are now served from the results cache on subsequent requests instead of being re-executed against the source, reducing planning and query latency for these patterns.

Faster Repeated Queries on Multi-File Datasets​

Object store datasets using parquet now cache Parquet footer statistics across queries. This reduces the frequency of Parquet footer parsing during planning, subsequently heavily reducing planning latency for certain query patterns (e.g. COUNT(*)).

Embedding Parameter Regression Fixes​

v2.1.0 introduced explicit definitions across embedding component parameters (i.e. .embeddings[].params). This introduced regressions for AWS Bedrock parameters truncation and truncation_mode that caused panics.

Contributors​

• @bjchambers
• @Jeadie
• @phillipleblanc
• @sgrebnov

Breaking Changes​

No breaking changes.

Cookbook Updates​

No new cookbook recipes.

The Spice Cookbook includes more than 100 recipes to help you get started with Spice quickly and easily.

Upgrading​

To upgrade to v2.1.1, use one of the following methods:

CLI:

spice upgrade

Homebrew:

brew upgrade spiceai/spiceai/spice

Docker:

Pull the spiceai/spiceai:2.1.1 image:

docker pull spiceai/spiceai:2.1.1

For available tags, see DockerHub.

Helm:

helm repo update
helm upgrade spiceai spiceai/spiceai --version 2.1.1

AWS Marketplace:

Spice is available in the AWS Marketplace.

What's Changed​

Changelog​

• fix(cache): cache empty (zero-row) SQL result sets by @bjchambers in #11699
• fix(cayenne): zero-row append refresh fails with "No such file or directory" by @sgrebnov in #11710
• feat(file): cache ListingTable file statistics to avoid per-query footer re-parse by @phillipleblanc in #11793
• fix(embeddings): restore params broken by #10853 by @Jeadie in #11788
• fix(cayenne): partitioned datasets deadlock against the global encode budget and never become ready by @Jeadie in #11825

Full Changelog: https://github.com/spiceai/spiceai/compare/v2.1.0...v2.1.1

Announcing the release of Spice v1.11.3! 🛠️

Spice v1.11.3 is a patch release fixing schema consistency issues in the S3 and FlightSQL data connectors, improving CDC cache invalidation, and enhancing the HTTP data connector's error handling and response metadata.

What's New in v1.11.3​

S3 Data Connector Fix​

Fixed an issue where queries using metadata columns (location, last_modified, size) on S3 datasets produced Input field name does not match with the projection expression errors (#9647). This occurred when projecting metadata columns with filters or scalar functions (e.g., SELECT lower(location) FROM table WHERE location = '...'), and when projection returned no matching files.

FlightSQL Schema Consistency​

Fixed an issue where the Flight SQL JDBC driver returned Unsupported ArrowType Utf8View errors when performing ::TEXT type casts (#9253). The FlightSQL endpoint now maps view types (e.g., Utf8View, BinaryView) to their non-view equivalents, ensuring compatibility with JDBC and ODBC clients.

CDC Cache Invalidation​

Fixed an issue where the SQL results cache was invalidated on every change stream poll, even when zero records were returned (#9472). This caused near-total cache miss rates for datasets using refresh_mode: changes (e.g., DynamoDB Streams), effectively rendering the cache useless. Cache invalidation now only occurs when a change batch contains actual data changes.

HTTP Data Connector Improvements​

• HTTP error responses (e.g., 5xx) are now excluded from the cache, preventing transient server errors from polluting cached results.
• Added a response_headers column (Map type) to HTTP responses, providing access to response header metadata in query results.

Contributors​

• @krinart
• @sgrebnov

Breaking Changes​

No breaking changes.

Cookbook Updates​

No new cookbook recipes.

The Spice Cookbook includes 86 recipes to help you get started with Spice quickly and easily.

Upgrading​

To upgrade to v1.11.3, use one of the following methods:

CLI:

spice upgrade

Homebrew:

brew upgrade spiceai/spiceai/spice

Docker:

Pull the spiceai/spiceai:1.11.3 image:

docker pull spiceai/spiceai:1.11.3

For available tags, see DockerHub.

Helm:

helm repo update
helm upgrade spiceai spiceai/spiceai --version 1.11.3

AWS Marketplace:

Spice is available in the AWS Marketplace.

What's Changed​

Changelog​

• fix(s3): Fix metadata column schema mismatches in projected queries by @sgrebnov in #9664
• s3_metadata_columns tests: include test for location outside table prefix by @sgrebnov in #9676
• Fix Flight SQL schema consistency: expand view types and verify field names by @sgrebnov in #9438
• Improve CDC cache invalidation by @krinart in #9651
• Skip caching http error response + add response_headers by @krinart in #9670

Full Changelog: https://github.com/spiceai/spiceai/compare/v1.11.2...v1.11.3

Announcing the release of Spice v1.11.0-stable! ⚡

In Spice v1.11.0, Spice Cayenne reaches Beta status with acceleration snapshots, Key-based deletion vectors, and Amazon S3 Express One Zone support. DataFusion has been upgraded to v51 along with Arrow v57.2, and iceberg-rust v0.8.0. v1.11 adds several DynamoDB & DynamoDB Streams improvements such as JSON nesting, and adds significant improvements to Distributed Query with active-active schedulers and mTLS for enterprise-grade high-availability and secure cluster communication.

This release also adds new SMB, NFS, and ScyllaDB Data Connectors (Alpha), Prepared Statements with full SDK support (gospice, spice-rs, spice-dotnet, spice-java, spice.js, and spicepy), Google LLM Support for expanded AI inference capabilities, and significant improvements to caching, observability, and Hash Indexing for Arrow Acceleration.

What's New in v1.11.0​

Spice Cayenne Accelerator Reaches Beta​

Spice Cayenne has been promoted to Beta status with acceleration snapshots support and numerous performance and stability improvements.

Key Enhancements:

• Key-based Deletion Vectors: Improved deletion vector support using key-based lookups for more efficient data management and faster delete operations. Key-based deletion vectors are more memory-efficient than positional vectors for sparse deletions.
• S3 Express One Zone Support: Store Cayenne data files in S3 Express One Zone for single-digit millisecond latency, ideal for latency-sensitive query workloads that require persistence.

Improved Reliability:

• Resolved FuturesUnordered reentrant drop crashes
• Fixed memory growth issues related to Vortex metrics allocation
• Metadata catalog now properly respects cayenne_file_path location
• Added warnings for unparseable configuration values

For more details, refer to the Cayenne Documentation.

DataFusion v51 Upgrade​

Apache DataFusion has been upgraded to v51, bringing significant performance improvements, new SQL features, and enhanced observability.

Performance Improvements:

• Faster CASE Expression Evaluation: Expressions now short-circuit earlier, reuse partial results, and avoid unnecessary scattering, speeding up common ETL patterns
• Better Defaults for Remote Parquet Reads: DataFusion now fetches the last 512KB of Parquet files by default, typically avoiding 2 I/O requests per file
• Faster Parquet Metadata Parsing: Leverages Arrow 57's new thrift metadata parser for up to 4x faster metadata parsing

New SQL Features:

• SQL Pipe Operators: Support for |> syntax for inline transforms
• DESCRIBE <query>: Returns the schema of any query without executing it
• Named Arguments in SQL Functions: PostgreSQL-style param => value syntax for scalar, aggregate, and window functions
• Decimal32/Decimal64 Support: New Arrow types supported including aggregations like SUM, AVG, and MIN/MAX

Example pipe operator:

SELECT * FROM t
|> WHERE a > 10
|> ORDER BY b
|> LIMIT 5;

Improved Observability:

• Improved EXPLAIN ANALYZE Metrics: New metrics including output_bytes, selectivity for filters, reduction_factor for aggregates, and detailed timing breakdowns

Arrow 57.2 Upgrade​

Apache Arrow has been upgraded to v57.2, bringing major performance improvements and new capabilities.

Key Features:

• 4x Faster Parquet Metadata Parsing: A rewritten thrift metadata parser delivers up to 4x faster metadata parsing, especially beneficial for low-latency use cases and files with large amounts of metadata
• Parquet Variant Support: Experimental support for reading and writing the new Parquet Variant type for semi-structured data, including shredded variant values
• Parquet Geometry Support: Read and write support for Parquet Geometry types (GEOMETRY and GEOGRAPHY) with GeospatialStatistics
• New arrow-avro Crate: Efficient conversion between Apache Avro and Arrow RecordBatches with projection pushdown and vectorized execution support

DynamoDB Connector Enhancements​

• Added JSON nesting for DynamoDB Streams
• Improved batch deletion handling

Distributed Query Improvements​

High Availability Clusters: Spice now supports running multiple active schedulers in an active/active configuration for production deployments. This eliminates the scheduler as a single point of failure and enables graceful handling of node failures.

• Multiple schedulers run simultaneously, each capable of accepting queries
• Schedulers coordinate via a shared S3-compatible object store
• Executors discover all schedulers automatically
• A load balancer distributes client queries across schedulers

Example HA configuration:

runtime:
  scheduler:
    state_location: s3://my-bucket/spice-cluster
    params:
      region: us-east-1

mTLS Verification: Cluster communication between scheduler and executors now supports mutual TLS verification for enhanced security.

Credential Propagation: S3, ABFS, and GCS credentials are now automatically propagated to executors in cluster mode, enabling access to cloud storage across the distributed query cluster.

Improved Resilience:

• Exponential backoff for scheduler disconnection recovery
• Increased gRPC message size limit from 16MB to 100MB for large query plans
• HTTP health endpoint for cluster executors
• Automatic executor role inference when --scheduler-address is provided

For more details, refer to the Distributed Query Documentation.

iceberg-rust v0.8.0 Upgrade​

Spice has been upgraded to iceberg-rust v0.8.0, bringing improved Iceberg table support.

Key Features:

• V3 Metadata Support: Full support for Iceberg V3 table metadata format
• INSERT INTO Partitioned Tables: DataFusion integration now supports inserting data into partitioned Iceberg tables
• Improved Delete File Handling: Better support for position and equality delete files, including shared delete file loading and caching
• SQL Catalog Updates: Implement update_table and register_table for SQL catalog
• S3 Tables Catalog: Implement update_table for S3 Tables catalog
• Enhanced Arrow Integration: Convert Arrow schema to Iceberg schema with auto-assigned field IDs, _file column support, and Date32 type support

Acceleration Snapshots​

Acceleration snapshots enable point-in-time recovery and data versioning for accelerated datasets. Snapshots capture the state of accelerated data at specific points, allowing for fast bootstrap recovery and rollback capabilities.

Key Features:

• Flexible Triggers: Configure when snapshots are created based on time intervals or stream batch counts
• Automatic Compaction: Reduce storage overhead by compacting older snapshots (DuckDB only)
• Bootstrap Integration: Snapshots can reset cache expiry on load for seamless recovery (DuckDB with Caching refresh mode)
• Smart Creation Policies: Only create snapshots when data has actually changed

Example configuration:

datasets:
  - from: s3://my-bucket/data.parquet
    name: my_dataset
    acceleration:
      enabled: true
      engine: cayenne
      mode: file
      snapshots: enabled
      snapshots_trigger: time_interval
      snapshots_trigger_threshold: 1h
      snapshots_creation_policy: on_changed

Snapshots API and CLI: New API endpoints and CLI commands for managing snapshots programmatically.

CLI Commands:

# List all snapshots for a dataset
spice acceleration snapshots taxi_trips

# Get details of a specific snapshot
spice acceleration snapshot taxi_trips 3

# Set the current snapshot for rollback (requires runtime restart)
spice acceleration set-snapshot taxi_trips 2

HTTP API Endpoints:

For more details, refer to the Acceleration Snapshots Documentation.

Caching Acceleration Mode Improvements​

The Caching Acceleration Mode introduced in v1.10.0 has received significant performance optimizations and reliability fixes in this release.

Performance Optimizations:

• Non-blocking Cache Writes: Cache misses no longer block query responses. Data is written to the cache asynchronously after the query returns, reducing query latency for cache miss scenarios.
• Batch Cache Writes: Multiple cache entries are now written in batches rather than individually, significantly improving write throughput for high-volume cache operations.

Reliability Fixes:

• Correct SWR Refresh Behavior: The stale-while-revalidate (SWR) pattern now correctly refreshes only the specific entries that were accessed instead of refreshing all stale rows in the dataset. This prevents unnecessary source queries and reduces load on upstream data sources.
• Deduplicated Refresh Requests: Fixed an issue where JSON array responses could trigger multiple redundant refresh operations. Refresh requests are now properly deduplicated.
• Fixed Cache Hit Detection: Resolved an issue where queries that didn't include fetched_at in their projection would always result in cache misses, even when cached data was available.
• Unfiltered Query Optimization: SELECT * queries without filters now return cached data directly without unnecessary filtering overhead.

For more details, refer to the Caching Acceleration Mode Documentation.

Prepared Statements​

Improved Query Performance and Security: Spice now supports prepared statements, enabling parameterized queries that improve both performance through query plan caching and security by preventing SQL injection attacks.

Key Features:

• Query Plan Caching: Prepared statements cache query plans, reducing planning overhead for repeated queries
• SQL Injection Prevention: Parameters are safely bound, preventing SQL injection vulnerabilities
• Arrow Flight SQL Support: Full prepared statement support via Arrow Flight SQL protocol

SDK Support:

Example (Go):

import "github.com/spiceai/gospice/v8"

client, _ := spice.NewClient()
defer client.Close()

// Parameterized query with typed parameters
results, _ := client.SqlWithParams(ctx,
    "SELECT * FROM products WHERE price > $1 AND category = $2",
    spice.Float64Param(10.0),
    spice.StringParam("electronics"),
)

Example (Java):

import ai.spice.SpiceClient;
import ai.spice.Param;
import org.apache.arrow.adbc.core.ArrowReader;

try (SpiceClient client = new SpiceClient()) {
    // With automatic type inference
    ArrowReader reader = client.queryWithParams(
        "SELECT * FROM products WHERE price > $1 AND category = $2",
        10.0, "electronics");

    // With explicit typed parameters
    ArrowReader reader = client.queryWithParams(
        "SELECT * FROM products WHERE price > $1 AND category = $2",
        Param.float64(10.0),
        Param.string("electronics"));
}

For more details, refer to the Parameterized Queries Documentation.

Spice Java SDK v0.5.0​

Parameterized Query Support for Java: The Spice Java SDK v0.5.0 introduces parameterized queries using ADBC (Arrow Database Connectivity), providing a safer and more efficient way to execute queries with dynamic parameters.

Key Features:

• SQL Injection Prevention: Parameters are safely bound, preventing SQL injection vulnerabilities
• Automatic Type Inference: Java types are automatically mapped to Arrow types (e.g., double → Float64, String → Utf8)
• Explicit Type Control: Use the new Param class with typed factory methods (Param.int64(), Param.string(), Param.decimal128(), etc.) for precise control over Arrow types
• Updated Dependencies: Apache Arrow Flight SQL upgraded to 18.3.0, plus new ADBC driver support

Example:

import ai.spice.SpiceClient;
import ai.spice.Param;

try (SpiceClient client = new SpiceClient()) {
    // With automatic type inference
    ArrowReader reader = client.queryWithParams(
        "SELECT * FROM taxi_trips WHERE trip_distance > $1 LIMIT 10",
        5.0);

    // With explicit typed parameters for precise control
    ArrowReader reader = client.queryWithParams(
        "SELECT * FROM orders WHERE order_id = $1 AND amount >= $2",
        Param.int64(12345),
        Param.decimal128(new BigDecimal("99.99"), 10, 2));
}

Maven:

<dependency>
  <groupId>ai.spice</groupId>
  <artifactId>spiceai</artifactId>
  <version>0.5.0</version>
</dependency>

For more details, refer to the Spice Java SDK Repository.

Google LLM Support​

Expanded AI Provider Support: Spice now supports Google embedding and chat models via the Google AI provider, expanding the available LLM options for AI inference workloads alongside existing providers like OpenAI, Anthropic, and AWS Bedrock.

Key Features:

• Google Chat Models: Access Google's Gemini models for chat completions
• Google Embeddings: Generate embeddings using Google's text embedding models
• Unified API: Use the same OpenAI-compatible API endpoints for all LLM providers

Example spicepod.yaml configuration:

models:
  - from: google:gemini-2.0-flash
    name: gemini
    params:
      google_api_key: ${secrets:GOOGLE_API_KEY}

embeddings:
  - from: google:text-embedding-004
    name: google_embeddings
    params:
      google_api_key: ${secrets:GOOGLE_API_KEY}

For more details, refer to the Google LLM Documentation (see docs PR #1286).

URL Tables​

Query data sources directly via URL in SQL without prior dataset registration. Supports S3, Azure Blob Storage, and HTTP/HTTPS URLs with automatic format detection and partition inference.

Supported Patterns:

• Single files: SELECT * FROM 's3://bucket/data.parquet'
• Directories/prefixes: SELECT * FROM 's3://bucket/data/'
• Glob patterns: SELECT * FROM 's3://bucket/year=*/month=*/data.parquet'

Key Features:

• Automatic file format detection (Parquet, CSV, JSON, etc.)
• Hive-style partition inference with filter pushdown
• Schema inference from files
• Works with both SQL and DataFrame APIs

Example with hive partitioning:

-- Partitions are automatically inferred from paths
SELECT * FROM 's3://bucket/data/' WHERE year = '2024' AND month = '01'

Enable via spicepod.yml:

runtime:
  params:
    url_tables: enabled

Cluster Mode Async Query APIs (experimental)​

New asynchronous query APIs for long-running queries in cluster mode:

• /v1/queries endpoint: Submit queries and retrieve results asynchronously

OpenTelemetry Improvements​

Unified Telemetry Endpoint: OTel metrics ingestion has been consolidated to the Flight port (50051), simplifying deployment by removing the separate OTel port (50052). The push-based metrics exporter continues to support integration with OpenTelemetry collectors.

Note: This is a breaking change. Update your configurations if you were using the dedicated OTel port 50052. Internal cluster communication now uses port 50052 exclusively.

Observability Improvements​

Enhanced Dashboards: Updated Grafana and Datadog example dashboards with:

• Snapshot monitoring widgets
• Improved accelerated datasets section
• Renamed ingestion lag charts for clarity

Additional Histogram Buckets: Added more buckets to histogram metrics for better latency distribution visibility.

For more details, refer to the Monitoring Documentation.

Hash Indexing for Arrow Acceleration (experimental)​

Arrow-based accelerations now support hash indexing for faster point lookups on equality predicates. Hash indexes provide O(1) average-case lookup performance for columns with high cardinality.

Features:

• Primary key hash index support
• Secondary index support for non-primary key columns
• Composite key support with proper null value handling

Example configuration:

datasets:
  - from: postgres:users
    name: users
    acceleration:
      enabled: true
      engine: arrow
      primary_key: user_id
      indexes:
        '(tenant_id, user_id)': unique  # Composite hash index

For more details, refer to the Hash Index Documentation.

SMB and NFS Data Connectors​

Network-Attached Storage Connectors: New data connectors for SMB (Server Message Block) and NFS (Network File System) protocols enable direct federated queries against network-attached storage without requiring data movement to cloud object stores.

Key Features:

• SMB Protocol Support: Connect to Windows file shares and Samba servers with authentication support
• NFS Protocol Support: Connect to Unix/Linux NFS exports for direct data access
• Federated Queries: Query Parquet, CSV, JSON, and other file formats directly from network storage with full SQL support
• Acceleration Support: Accelerate data from SMB/NFS sources using DuckDB, Spice Cayenne, or other accelerators

Example spicepod.yaml configuration:

datasets:
  # SMB share
  - from: smb://fileserver/share/data.parquet
    name: smb_data
    params:
      smb_username: ${secrets:SMB_USER}
      smb_password: ${secrets:SMB_PASS}

  # NFS export
  - from: nfs://nfsserver/export/data.parquet
    name: nfs_data

For more details, refer to the Data Connectors Documentation.

ScyllaDB Data Connector​

A new data connector for ScyllaDB, the high-performance NoSQL database compatible with Apache Cassandra. Query ScyllaDB tables directly or accelerate them for faster analytics.

Example configuration:

datasets:
  - from: scylladb:my_keyspace.my_table
    name: scylla_data
    acceleration:
      enabled: true
      engine: duckdb

For more details, refer to the ScyllaDB Data Connector Documentation.

Flight SQL TLS Connection Fixes​

TLS Connection Support: Fixed TLS connection issues when using grpc+tls:// scheme with Flight SQL endpoints. Added support for custom CA certificate files via the new flightsql_tls_ca_certificate_file parameter.

Developer Experience Improvements​

• Turso v0.3.2 Upgrade: Upgraded Turso accelerator for improved performance and reliability
• Rust 1.91 Upgrade: Updated to Rust 1.91 for latest language features and performance improvements
• Spice Cloud CLI: Added spice cloud CLI commands for cloud deployment management
• Improved Spicepod Schema: Improved JSON schema generation for better IDE support and validation
• Acceleration Snapshots: Added configurable snapshots_create_interval for periodic acceleration snapshots independent of refresh cycles
• Tiered Caching with Localpod: The Localpod connector now supports caching refresh mode, enabling multi-layer acceleration where a persistent cache feeds a fast in-memory cache
• GitHub Data Connector: Added workflows and workflow runs support for GitHub repositories
• NDJSON/LDJSON Support: Added support for Newline Delimited JSON and Line Delimited JSON file formats

Additional Improvements & Bug Fixes​

• Model Listing: New functionality to list available models across multiple AI providers
• DuckDB Partitioned Tables: Primary key constraints now supported in partitioned DuckDB table mode
• Post-refresh Sorting: New on_refresh_sort_columns parameter for DuckDB enables data ordering after writes
• Improved Install Scripts: Removed jq dependency and improved cross-platform compatibility
• Better Error Messages: Improved error messaging for bucket UDF arguments and deprecated OpenAI parameters
• Reliability: Fixed DynamoDB IAM role authentication with new dynamodb_auth: iam_role parameter
• Reliability: Fixed cluster executors to use scheduler's temp_directory parameter for shuffle files
• Reliability: Initialize secrets before object stores in cluster executor mode
• Reliability: Added page-level retry with backoff for transient GitHub GraphQL errors
• Performance: Improved statistics for rewritten DistributeFileScanOptimizer plans
• Developer Experience: Added max_message_size configuration for Flight service

Contributors​

• @ewgenius
• @Jeadie
• @krinart
• @lukekim
• @mach-kernel
• @peasee
• @phillipleblanc
• @sgrebnov

Breaking Changes​

OTel Ingestion Port Change​

OTel ingestion has been moved to the Flight port (50051), removing the separate OTel port 50052. Port 50052 is now used exclusively for internal cluster communication. Update your configurations if you were using the dedicated OTel port.

Distributed Query Cluster Mode Requires mTLS​

Distributed query cluster mode now requires mTLS for secure communication between cluster nodes. This is a security enhancement to prevent unauthorized nodes from joining the cluster and accessing secrets.

Migration Steps:

1. Generate certificates using spice cluster tls init and spice cluster tls add
2. Update scheduler and executor startup commands with --node-mtls-* arguments
3. For development/testing, use --allow-insecure-connections to opt out of mTLS

Renamed CLI Arguments:

Removed CLI Arguments:

• --cluster-api-key: Replaced by mTLS authentication

Cookbook Updates​

New ScyllaDB Data Connector Recipe: New recipe demonstrating how to use the ScyllaDB Data Connector. See ScyllaDB Data Connector Recipe for details.

New SMB Data Connector Recipe: New recipe demonstrating how to use the SMB Data Connector. See SMB Data Connector Recipe for details.

The Spice Cookbook includes 86 recipes to help you get started with Spice quickly and easily.

Upgrading​

To upgrade to v1.11.0, use one of the following methods:

CLI:

spice upgrade

Homebrew:

brew upgrade spiceai/spiceai/spice

Docker:

Pull the spiceai/spiceai:1.11.0 image:

docker pull spiceai/spiceai:1.11.0

For available tags, see DockerHub.

Helm:

helm repo update
helm upgrade spiceai spiceai/spiceai --version 1.11.0

AWS Marketplace:

Spice is available in the AWS Marketplace.

Dependencies​

• DataFusion: Upgraded to v51 (release notes)
• Arrow: Upgraded to v57.2 (release notes)
• iceberg-rust: Upgraded to v0.8.0 (release notes)

What's Changed​

Changelog​

• OTel exporter for push metrics by @lukekim in #8442
• fix: Update benchmark snapshots by @app/github-actions in #8448
• Add TPCH append tests to scheduled dispatch workflow by @sgrebnov in #8451
• Add snapshot creation logging by @krinart in #8469
• Fix PeriodicReader panic by @krinart in #8471
• Benchmarks: increase readiness timeout for turso acceleration (TPC-H) by @sgrebnov in #8470
• fix: Pin CUDA build actions to commits by @peasee in #8477
• Add Criterion benchmarking to chunking crate. by @Jeadie in #8431
• DuckDB agg pushdown: gate behind accelerator parameter by @mach-kernel in #8474
• Rename aggregate_pushdown_optimization -> optimizer_duckdb_aggregate_pushdown by @ewgenius in #8485
• Handle throttling exception for DynamoDB streams by @phillipleblanc in #8492
• docs: Add release notes by @peasee in #8478
• Update spicepod.schema.json by @app/github-actions in #8496
• Move 'test_projection_pushdown' to runtime-datafusion by @Jeadie in #8490
• Fix OTEL metrics HTTP exporter client setup by @phillipleblanc in #8489
• Update endgame to include new caching accelerator cookbook by @phillipleblanc in #8487
• DynamoDB tests and fixes by @lukekim in #8491
• Align make lint-rust-fix with make lint-rust by @Jeadie in #8499
• fix: Remove unused Cayenne parameters by @peasee in #8500
• Force task history captured_plan outputs to be captured even if they would be filtered out otherwise by @phillipleblanc in #8501
• release: post-release updates by @peasee in #8503
• CI: Fix E2E models dispatch by @mach-kernel in #8505
• Use an isolated Tokio runtime for refresh tasks that is separate from the main query API by @phillipleblanc in #8504
• Update openapi.json by @app/github-actions in #8512
• Update dependencies by @phillipleblanc in #8513
• fix: Avoid double hashing cache key by @peasee in #8511
• fix: Eagerly drop cached records for results larger than max by @peasee in #8516
• feat: Support vortex zstd compressor by @peasee in #8515
• warning if column is defined in spicepod but is non-existant by @Jeadie in #8498
• Return summarized spicepods from /v1/spicepods by @phillipleblanc in #8404
• Fix for idle DynamoDB Stream by @krinart in #8506
• Use Datafusion::Plan over Datafusion::Internal for user-facing search errors. by @Jeadie in #8484
• DDB Streams Integration Test + Memory Acceleration + Improved Warning by @krinart in #8520
• Upgrade to gospice v8 by @lukekim in #8524
• Vortex file format for object store by @Jeadie in #8525
• docs: Add missing cookbooks to endgame, focus area section by @peasee in #8527
• ListingTableConnector: Drop partition columns that reoccur in file schema by @mach-kernel in #8519
• fix(cluster): initialize secrets before object stores in executor by @sgrebnov in #8532
• Use separate Tokio runtime for SWR refreshes by @phillipleblanc in #8530
• fix: SQL Results Cache SWR triggers incorrect cache miss metric by @phillipleblanc in #8529
• feat: testoperator: OTLP streaming metrics / connect to existing instance / infinite mode / query timeout by @phillipleblanc in #8537
• Update openapi.json by @app/github-actions in #8518
• Add better attributes for search testing by @Jeadie in #8531
• fix: Improve Cayenne errors, ID selection for table/partition creation by @peasee in #8523
• Percent-encode Kubernetes secret name path segment by @phillipleblanc in #8522
• Initial parameterisation of Search integration tests by @Jeadie in #8066
• fix: Add warning when multiple partitions are defined for the same table by @peasee in #8540
• Add DuckDB file-mode support for search test parameterization. by @Jeadie in #8541
• fix: Add recursion depth limits to prevent DoS via deeply nested data (DynamoDB + S3 Vectors) by @phillipleblanc in #8544
• Remove the clippy::too_many_lines lint by @phillipleblanc in #8549
• feat: Add spice cluster tls commands by @phillipleblanc in #8550
• Move OTel ingestion to Flight port, remove separate OTel port 50052 by @phillipleblanc in #8551
• Move verbose tool init messages to trace by @phillipleblanc in #8552
• Add SANs to spice cluster tls add certificates by @phillipleblanc in #8554
• Add cpu, gpu, and memory to telemetry by @lukekim in #8483
• feat: Add workflows and workflow runs to GitHub Data Connector by @peasee in #8548
• Add S3Vectors option for paramterised search tests by @Jeadie in #8555
• Distributed query: TLS + API key by @mach-kernel in #8468
• Refactor RRF SQL to use LogicalPlanBuilder by @Jeadie in #7968
• Require mTLS for distributed query cluster mode by @phillipleblanc in #8580
• Fix stats for rewritten DistributeFileScanOptimizer plans by @mach-kernel in #8581
• Bump actions/cache from 4.3.0 to 5.0.1 by @app/dependabot in #8573
• Show user-friendly error on empty DDB table by @krinart in #8586
• Add (Deprecated) labels to deprecated spice sql params by @krinart in #8588
• Fix kafka warning when security.protocol is set to PLAINTEXT by @krinart in #8587
• Upgrade dependencies by @phillipleblanc in #8593
• Release notes for v1.10.1 by @Jeadie in #8568
• Run search benchmarks twice a week. by @Jeadie in #8592
• Add cayenne data accelerator by @Jeadie in #8553
• SQL allowlist for tools: sql, list_datasets, search, table_schema by @Jeadie in #8449
• use rstest for llms integration tests by @Jeadie in #8566
• Post v1.10.1 housekeeping by @Jeadie in #8600
• Add checklist for SDK publication in end_game.md by @Jeadie in #8602
• Fix IMAGE_TAG assignment for Docker compatibility by @Jeadie in #8599
• Modify cluster arguments to spiced for UX review by @phillipleblanc in #8603
• Update QA analytics with new release data by @Jeadie in #8601
• Remove 'tract-core' dependency. by @Jeadie in #8605
• Google embedding and chat models. by @Jeadie in #8423
• Fix test_github_workflows integration test by @sgrebnov in #8607
• Update openapi.json by @app/github-actions in #8604
• ci: Upload artifacts to MinIO eagerly after each build step by @phillipleblanc in #8615
• fix: SQLite accelerator decimal/date handling by @phillipleblanc in #8606
• Configure mTLS for executor-to-executor gRPC connections by @sgrebnov in #8617
• feat: Enable localpod with caching mode accelerator for tiered caching by @phillipleblanc in #8621
• Add Cayenne S3 Express One Zone support for data files by @lukekim in #8502
• Add snapshot interval for acceleration snapshots by @phillipleblanc in #8627
• Add dataset_load_parallelism parameter to spicepod.yml by @peasee in #8630
• Json Nesting for DynamoDB by @krinart in #8623
• Restore deprecated open-telemetry flag in spiced by @phillipleblanc in #8629
• Implement batching for Kafka/Debezium + null Decimal handling by @krinart in #8622
• fix: Status field in /v1/datasets & /v1/models by @lukekim in #8633
• Add Spice test operator improvements by @sgrebnov in #8625
• Add v1.10.2 release notes by @sgrebnov in #8640
• Align object-store vortex with runtime feature flagging by @Jeadie in #8620
• Extended LLM & search tests on cron by @Jeadie in #8624
• Test-operator: emit main metrics as part of load tests by @sgrebnov in #8639
• Build a local docker image from an existing Spice binary by @phillipleblanc in #8619
• fix: Use runtime-rate-control for GitHub Data Connector by @peasee in #8638
• Upgrade dependencies by @phillipleblanc in #8655
• Bump headers-accept from 0.1.4 to 0.3.0 by @app/dependabot in #8644
• Update openapi.json by @app/github-actions in #8637
• Update SECURITY.md - Include v1.10.2 by @sgrebnov in #8661
• Serialize acceleration snapshots with refresh writes by @phillipleblanc in #8652
• Update AI Installation test to use minilm_l6_v2 by @sgrebnov in #8659
• Fixes for search integration test CI by @Jeadie in #8656
• fix: Use a GitHub rate controller per auth context by @peasee in #8662
• fix: Update Search integration test snapshots by @app/github-actions in #8654
• Make E2E Test Release Installation (AI, Local HF model) test more robust by @sgrebnov in #8666
• Fix issue with location predicate for custom S3 endpoints + regression integration test by @phillipleblanc in #8668
• fix: Validate schema match before projection pushdown in UnionProjectionPushdownOptimizer by @phillipleblanc in #8669
• Proper batch commit for kafka/debezium by @krinart in #8671
• Improve spicepod json schema generation by @ewgenius in #8547
• Start the anonymous telemetry exporter asynchronously by @phillipleblanc in #8679
• fix: Move enforce-pulls to hosted runner by @phillipleblanc in #8686
• Update QA analytics with 1.10.2 release data by @sgrebnov in #8667
• fix: Azure does not support suffix range requests by @phillipleblanc in #8685
• Remove spicepod-validator cargo build from build-dev target by @Jeadie in #8684
• fix: Update test snapshots by @app/github-actions in #8680
• fix: Update Search integration test snapshots by @app/github-actions in #8681
• SMB and NFS Data Connectors by @lukekim in #8674
• Upgrade to openai-async v0.32 by @lukekim in #8635
• v1.10.3 release notes by @phillipleblanc in #8693
• Upgrade dependencies by @phillipleblanc in #8704
• fix: Support NDJSON and LDJSON by @lukekim in #8649
• move OpenAI overrides to non-prefixed by @Jeadie in #8678
• Update Google LLM param: google_dimensions -> dimensions by @Jeadie in #8677
• Make cluster mTLS optional with insecure flag by @phillipleblanc in #8703
• Revert "fix: Move enforce-pulls to hosted runner (#8686)" by @phillipleblanc in #8709
• Initial 'testoperator run text-to-sql' by @Jeadie in #8618
• Add support for abfss by @krinart in #8706
• Add testoperator TPCH dispatch for ABFS with hierarchical namespace disabled + versioning enabled by @phillipleblanc in #8711
• Update openapi.json by @app/github-actions in #8692
• cluster: validate --role argument by @phillipleblanc in #8717
• Upgrade to Turso v0.3.2 by @lukekim in #8716
• Rename --insecure to --allow-insecure-connections to be consistent with existing naming by @lukekim in #8720
• Remove 'testoperator run http-consistency/http-overhead' by @Jeadie in #8708
• refactor: Remove cluster feature flag by @phillipleblanc in #8718
• Docs: Distributed query ADR by @mach-kernel in #8608
• Use model.datasets to allowlist on tools by @Jeadie in #8714
• cluster: quality of life improvements to starting cluster mode locally by @phillipleblanc in #8719
• Docs: Ballista extension ADR by @mach-kernel in #8616
• Improve deprecation messages when going from prefixed -> non-prefixed. by @Jeadie in #8724
• Remove tools from auto-defaults by @Jeadie in #8725
• Make distinct providers for vector spilling, vector partitioning. by @Jeadie in #8546
• cluster: default scheduler address port by @phillipleblanc in #8728
• Add Makefile targets for testoperator by @Jeadie in #8729
• text-to-sql dispatch in testoperator by @Jeadie in #8705
• DR-006: High Availability Distributed Query with Stateless Schedulers by @lukekim in #8721
• DR-007: mTLS for Distributed Query Cluster Communication by @lukekim in #8722
• SMB and NFS improvements by @lukekim in #8710
• fix: Cluster executors use scheduler's temp_directory for shuffle files by @phillipleblanc in #8733
• use 'max_message_size' in flight service too by @Jeadie in #8730
• Add page-level retry for transient GraphQL errors with backoff and increase GitHub rate limit buffer up to 100 by @ewgenius in #8726
• Make testoperator Dockerfile; CI to build docker image to ghcr.io. by @Jeadie in #8732
• cluster: UnionProjectionPushdownOptimizer: Add projection pushdown diagnostics for union children by @phillipleblanc in #8734
• Fix column projection order mismatch with location metadata columns by @phillipleblanc in #8738
• Fixes for testoperator. by @Jeadie in #8737
• Improve Cayenne Deletion Vectors with KeyBased support by @lukekim in #8713
• Fix testoperator_dispatch.yaml by @Jeadie in #8740
• Add spice cloud CLI commands by @lukekim in #8528
• Add FTP, NFS, & SMB TPCH SF1 spicepods by @lukekim in #8739
• Prepared Statements by @lukekim in #7588
• Schedule dispatch of testoperator run text-to-sql. by @Jeadie in #8745
• Fix minio for ai benchmark CI by @Jeadie in #8743
• Upgrade to Rust 1.91 by @phillipleblanc in #8749
• fix: Update benchmark snapshots by @app/github-actions in #8763
• Benchmarks: make row count validation skip logic configurable by scale factor, query set, and overrides by @sgrebnov in #8756
• Make benchmark tests more robust by @sgrebnov in #8766
• Add parameter to force using iam_role for DynamoDB by @krinart in #8767
• fix: Update Search integration test snapshots by @app/github-actions in #8735
• v1.10.4 release notes by @phillipleblanc in #8790
• Trace metrics export errors by @sgrebnov in #8791
• v1.10.4 SECURITY.md update by @phillipleblanc in #8800
• Add timezone database to Docker image to fix Cayenne acceleration panic by @sgrebnov in #8799
• Upgrade dependencies by @phillipleblanc in #8801
• Fix table_allowlist for table sampling and NSQL by @Jeadie in #8789
• Cayenne primary key on-conflict handling by @lukekim in #8788
• fix: Update benchmark snapshots by @app/github-actions in #8773
• fix: correctly identify deprecated openai_* parameters by @phillipleblanc in #8809
• fix: Update benchmark snapshots by @app/github-actions in #8812
• Use workspace version for cayenne crate by @phillipleblanc in #8811
• Don't CAST strings which breaks push down optimizer by @lukekim in #8810
• fix: Update benchmark snapshots by @app/github-actions in #8815
• Update async-openai to latest revision 4dcd633aad6f - brings fix for openai compatible model providers by @ewgenius in #8816
• Add auth/iam_role_source to DynamoDB connector by @krinart in #8808
• DynamoDB fixes: JSON nesting for Streams, proper batch deletions by @krinart in #8821
• Rough roadmap for 2026-2027 by @lukekim in #8805
• Release notes for v1.11.0-rc1 by @ewgenius in #8786
• Make S3V integration tests prepare_for_aws_tests more robust by @sgrebnov in #8820
• Bump rsa from 0.9.9 to 0.9.10 in the cargo group across 1 directory by @app/dependabot in #8819
• Add timezone database to Release and CUDA Docker images to fix Cayene panic by @sgrebnov in #8832
• fix: UnionProjectionPushdownOptimizer - Schema change during transform_down breaks parent nodes by @phillipleblanc in #8831
• Update grafana/datadog example dashboards by @krinart in #8833
• Add Dev bird bench as text-to-sql queryset in CI. by @Jeadie in #8753
• Update testoperator scheduler to use release/1.11 branch by @ewgenius in #8829
• Spice Cayenne fixes and test spicepods for Beta & RC by @lukekim in #8787
• testoperator dispatch all bird-bench database variants by @Jeadie in #8835
• feat: Improve column statistics handling with safe access and defaults by @phillipleblanc in #8836
• cluster: mTLS verification by @phillipleblanc in #8837
• fix: 8770: Unsupported ScalarFunctionExpr in ORDER BY by @lukekim in #8838
• Workflow tweaks by @lukekim in #8845
• Cayenne: metadata catalog should respect cayenne_file_path location by @sgrebnov in #8844
• Expand Cayenne feature coverage by @lukekim in #8848
• docs: HA distributed query decisions by @phillipleblanc in #8817
• fix(optimizer): Fix correctness issues in UnionProjectionPushdownOptimizer by @phillipleblanc in #8851
• Pin reqwest to 0.12.24 to fix HuggingFace embedding model download by @ewgenius in #8853
• Fix builds and pin to Ubuntu 22.04 by @lukekim in #8856
• Revert "Fix builds and pin to Ubuntu 22.04" by @lukekim in #8861
• Ensure setup Rust is run by @lukekim in #8862
• fix: Ubuntu 24.04+ renamed libaio1 to libaio1t64 by @lukekim in #8865
• Upgrade to Pulls with Spice v2 by @lukekim in #8866
• Add limit and configuration name to 'testoperator run text-to-sql' by @Jeadie in #8839
• PR check and test optimization by @lukekim in #8868
• Upgrade S3 Vectors SDK and improve test robustness by @lukekim in #8867
• [Testoperator] Query level and improved aggregate level for NSQL by @Jeadie in #8840
• Add docker build for private branches for ghcr.io/spiceai/spiceai-dev by @phillipleblanc in #8873
• Expand the data acceleration round-trip test coverage by @lukekim in #8855
• fix: Provide a better error for improper bucket UDF arguments by @peasee in #8849
• ScyllaDB Data Connector by @lukekim in #8827
• Use tokio-rusqlite for Spice Cayenne SQLite by @lukekim in #8857
• Cayenne: fix FuturesUnordered reentrant drop crash by @sgrebnov in #8863
• Bump github/codeql-action from 4.31.9 to 4.31.10 by @app/dependabot in #8884
• Bump golang.org/x/sys from 0.39.0 to 0.40.0 by @app/dependabot in #8881
• Bump github.com/spiceai/gospice/v8 from 8.0.0 to 8.0.1 by @app/dependabot in #8883
• Bump roaring from 0.11.2 to 0.11.3 by @app/dependabot in #8885
• Bump golang.org/x/mod from 0.31.0 to 0.32.0 by @app/dependabot in #8882
• Bump aws-sdk-s3 from 1.115.0 to 1.119.0 by @app/dependabot in #8887
• Bump libc from 0.2.177 to 0.2.180 by @app/dependabot in #8886
• Bump tokio-util from 0.7.17 to 0.7.18 by @app/dependabot in #8889
• Bump governor from 0.10.2 to 0.10.4 by @app/dependabot in #8888
• fix: flaky test test_concurrent_partition_creation by @phillipleblanc in #8898
• Update Cayenne snapshots for TPC-DS by @lukekim in #8890
• Add more buckets to histogram metrics by @krinart in #8850
• feat: Add HTTP health endpoint for cluster executors by @phillipleblanc in #8899
• feat: Implement model listing functionality for multiple providers by @lukekim in #8901
• feat: Initial HA schedulers distributed query implementation by @phillipleblanc in #8852
• fix: infer executor role from --scheduler-address when --role is omitted by @phillipleblanc in #8903
• Improve install scripts and remove jq dependency by @lukekim in #8847
• Benchmarks: sort PartitionedUnionExec children for deterministic snapshot comparison by @sgrebnov in #8877
• Cayenne: share VortexFileCache across partitions via CayenneContext by @sgrebnov in #8880
• Update ballista to add exponential backoff for scheduler disconnection by @phillipleblanc in #8905
• Configurably add BirdBench evidence to testoperator text-to-SQL. by @Jeadie in #8904
• Helm: Allow command override via values.yaml by @sgrebnov in #8906
• Fix distributed query gRPC message size limit (16MB -> 100MB) by @phillipleblanc in #8900
• OS specific setup actions by @lukekim in #8909
• Cayenne should warn if unable to parse configuration value by @sgrebnov in #8907
• Add snapshots widgets to example dashboard by @krinart in #8910
• Add quality criteria for the features by @krinart in #8897
• Improve Accelerated Datasets section for Grafana/Datadog dashboards by @krinart in #8915
• Use HTTP traceparent in NSQL to support concurrency in 'testoperator run text-to-SQL' by @Jeadie in #8912
• Remove setup for cc from integration_models.yml by @Jeadie in #8917
• Propagate Azure and GCS credentials to executors in cluster mode by @phillipleblanc in #8918
• Cayenne: fix memory growth due to vortex metrics allocation by @sgrebnov in #8908
• fix(caching): Deduplicate refresh requests for JSON array responses by @sgrebnov in #8921
• fix(caching): Return cached data directly for unfiltered queries (SELECT *) by @sgrebnov in #8919
• Correct MinIO path syntax for spiced download by @Jeadie in #8916
• Acceleration snapshots compaction + Improved Snapshots UX by @krinart in #8858
• Change base image from bookworm-slim to trixie-slim by @Jeadie in #8923
• Add testoperator run text-to-sql metrics from LogicalPlan by @Jeadie in #8895
• Fix spicepod dependencies in testoperator by @Jeadie in #8875
• Update copilot instructions for data correctness by @lukekim in #8922
• Add BootstrapStatus + Snapshot bootstrapping parallelization by @krinart in #8926
• fix: add missing feature-gate for AWS Secrets Manager error variant by @phillipleblanc in #8928
• refactor: make ConnectorParams fields public for external connectors by @phillipleblanc in #8929
• fix(caching): SWR refreshes only accessed entry instead of all stale rows by @sgrebnov in #8931
• Cayenne: include cayenne_metadata_dir to known params by @sgrebnov in #8933
• Rename Ingestion Lag chart in example dashboards by @krinart in #8932
• fix(caching): Fix HTTP caching always MISS when projection excludes fetched_at by @sgrebnov in #8930
• Reset expiry after snapshot bootstraping for Caching by @krinart in #8925
• Set use_ssl=false for sccache by @lukekim in #8945
• Hash indexing for Arrow Acceleration by @lukekim in #8924
• [Cayenne] Acceleration snapshots support by @lukekim in #7973
• perf(caching): Non-blocking cache writes on cache miss by @sgrebnov in #8948
• Update NSQL models by @lukekim in #8951
• Hash Index Key verification by @lukekim in #8949
• Add snapshots_creation_policy param by @krinart in #8954
• Remove candle & cudarc from non-models build by @lukekim in #8955
• Acceleration Snapshots API and CLI by @lukekim in #8934
• Ignore test for data_components arrow::indexed::test_primary_key_value_matches_batch by @Jeadie in #8962
• fix: Update benchmark snapshots by @app/github-actions in #8965
• Hash Index secondary index support by @lukekim in #8958
• fix: Support primary key constraints in partitioned DuckDB tables mode by @sgrebnov in #8966
• perf(caching): Batch cache writes by @sgrebnov in #8959
• CI perf optimizations by @lukekim in #8968
• Fix Makefile linting by @Jeadie in #8970
• Fixes in testoperator run text-to-sql. by @Jeadie in #8927
• implement Chat::as_sql for xAI anthropic by @Jeadie in #8957
• Fix duckdb_file_path in search integration test by @Jeadie in #8972
• fix: Update benchmark snapshots by @app/github-actions in #8971
• Maintenance updates to Anthropic API by @Jeadie in #8956
• Add CacheBackend Trait, implement pingora-lru, and add throughput tests by @lukekim in #8080
• fix: Update benchmark snapshots by @app/github-actions in #8974
• fix: Update benchmark snapshots by @app/github-actions in #8975
• Make accelerator shutdown more robust by @lukekim in #8969
• feat(duckdb): Add on_refresh_sort_columns for post-write data ordering (initial version) by @sgrebnov in #8964
• Proper handling for initial snapshot by @krinart in #8911
• fix: Remove --no-default-features from cargo-hack command in features workflow by @phillipleblanc in #8977
• build(deps): bump actions/cache from 5.0.1 to 5.0.2 by @app/dependabot in #8983
• build(deps): bump actions/checkout from 4 to 6 by @app/dependabot in #8982
• build(deps): bump actions/setup-go from 6.1.0 to 6.2.0 by @app/dependabot in #8984
• build(deps): bump github.com/olekukonko/tablewriter from 1.1.2 to 1.1.3 by @app/dependabot in #8979
• build(deps): bump github.com/klauspost/compress from 1.18.2 to 1.18.3 by @app/dependabot in #8980
• Add /v1/queries and Arrow Flight async APIs by @lukekim in #8946
• build(deps): bump Vampire/setup-wsl from 5 to 6 by @app/dependabot in #8981
• fix: Update Search integration test snapshots by @app/github-actions in #8973
• build(deps): bump insta from 1.46.0 to 1.46.1 by @app/dependabot in #8988
• build(deps): bump schemars from 1.1.0 to 1.2.0 by @app/dependabot in #8985
• fix: Update benchmark snapshots by @app/github-actions in #8978
• fix: Data correctness edge cases by @lukekim in #8953
• Correct MinIO path syntax for spiced download (Part 2) by @Jeadie in #8995
• Make .spice/data in search integration tests by @Jeadie in #8992
• fix: Hash index composite keys null values by @lukekim in #9001
• Update Cayenne status to Beta by @lukekim in #9002
• fix: Disable TPC-DS result validation (not yet supported) by @sgrebnov in #9004
• feat: Upgrade to DataFusion v51 and dependencies by @lukekim in #8864
• Improvements for snapshots_creation_policy by @krinart in #9003
• fix(ci): restore cached spicepod-validator binary instead of lookup-only by @phillipleblanc in #9007
• Update version by @krinart in #9010
• Update lock file - https://github.com/spiceai/spiceai/commit/53babbf07ca8c1c7b2e1da42ce58c465d9bc9276/
• fix: Enable Cayenne acceleration snapshots by @lukekim in #9020
• Add TPC-DS integration tests with S3 source and PostgreSQL acceleration by @phillipleblanc in #9006
• fix(tests): fix flaky/slow/failing unit tests by @phillipleblanc in #9009
• fix: Update benchmark snapshots for DF51 upgrade by @app/github-actions in #9008
• fix: add feature gate to rrf TEST_EMBEDDING_MODEL by @phillipleblanc in #9017
• fix: features check by @phillipleblanc in #9014
• URL table support by @lukekim in #9018
• ScyllaDB key filter by @lukekim in #8997
• fix: Schema mismatch when using column projection with HTTP caching by @phillipleblanc in #9021
• Add more tests for HTTP caching with columns selection by @sgrebnov in #9025
• HTTP cache snapshots: default to time_interval and fix snapshots_creation_policy: on_change by @sgrebnov in #9026
• Fix duplicate snapshot creation on startup by @sgrebnov in #9029
• Remove waiting for runtime to be ready before creating snapshot by @krinart in #9033
• Fix snapshot on_change policy to skip when no writes occurred by @sgrebnov in #9028
• Release notes for release release/1.11.0-rc.2 by @krinart in #9016
• ci: use arduino/setup-protoc for official protobuf compiler by @phillipleblanc in #9036
• ci: install unzip on aarch64 runner for arduino/setup-protoc by @phillipleblanc in #9038
• fix: don't fail release if upload to minio fails by @phillipleblanc in #9039
• Improve validation and logging for hash indexes by @lukekim in #9047
• Pin to ubuntu-22.04 by @lukekim in #9068
• Fix broken telemetry for testoperator by @krinart in #9054
• Fix release builds by @lukekim in #9069
• Spice 1.11.0-rc3 release notes by @krinart in #9070
• Update spicepod.schema.json by @app/github-actions in #9071
• Add missing protoc step to setup-cc action by @krinart in #9041
• Fix TLS connection for grpc+tls:// Flight SQL endpoints and add custom CA certificate support by @phillipleblanc in #9073
• Update 1.11.0-rc.3 release notes by @krinart in #9082
• Fix formula_1 and codebase_community in bird-bench by @Jeadie in #9000
• Cayenne S3 Express One Zone improvements by @lukekim in #9015
• Add zlib1g-dev to CI by @lukekim in #9052
• Upgrade Vortex with CASE-WHEN by @lukekim in #9051
• fix: Cayenne CatalogError handling for constraint violations by @lukekim in #9050
• Fix Docker build failing to copy shared libraries due to ldd output parsing by @phillipleblanc in #9058
• feat: Change /v1/sql and FlightSQL to use local execution in cluster mode by @phillipleblanc in #9055
• Remove unmaintained dependencies by @lukekim in #9045
• Enable cayenne + changes stream by @Jeadie in #9053
• feat(cli): add spice query command for async queries REPL by @phillipleblanc in #9057
• Remove unncessary allocations by @lukekim in #9059
• Add dataset_acceleration_size_bytes metric by @krinart in #9062
• Fix tracing of sql_query beneath tool_use::sample_data. by @Jeadie in #9043
• Basic script to run distributed spice by @Jeadie in #9049
• Add integration tests for Acceleration Snapshots by @krinart in #9067
• Upgrade CUDA toolkit to 12.6.0 by @sgrebnov in #9079
• Install required protoc dependency for CUDA build by @sgrebnov in #9080
• feat(cluster): add executor control stream heartbeat by @phillipleblanc in #9072
• feat: Fix async queries API and integrate Ballista shuffle improvements by @lukekim in #9075
• Remove models variant (now default) & Windows builds (use WSL) by @lukekim in #9063
• Cayenne: share upload semaphore across partitions to bound memory growth and optimize I/O by @sgrebnov in #9078
• Snowflake data connector - add snowflake_private_key parameter by @ewgenius in #9085
• Rewrite Go CLI in Rust by @phillipleblanc in #9061
• GCS Data Connector (Alpha) by @lukekim in #9084
• Skip 'latest' Docker tag for pre-release versions by @sgrebnov in #9077
• Add HTTP endpoints for acceleration snapshots API by @phillipleblanc in #9065
• Cayenne: Allow append mode with both primary_key and time_column by @sgrebnov in #9090
• Add Cluster Observability (Metrics+Dashboard) by @phillipleblanc in #9066
• proto for 'CayenneAccelerationExec' by @Jeadie in #9094
• Add 'anthropic-beta' header for structured outputs by @Jeadie in #9093
• Cayenne: refactor write path to use insert_into() as single entry point (part 1) by @sgrebnov in #9088
• Fix testoperator dispatch by @sgrebnov in #9097
• Fix setup-spiced GH action (_models suffix does not exist anymore) by @sgrebnov in #9102
• build(deps): bump github/codeql-action from 4.31.10 to 4.31.11 by @app/dependabot in #9108
• Remove DistributeFileScanOptimizer and UnionProjectionPushdownOptimizer & set target_partitions dynamically based on cluster capacity by @phillipleblanc in #9100
• build(deps): bump zip from 2.4.2 to 6.0.0 by @app/dependabot in #9111
• fix: Preserve query parameter order in HTTP connector to match filter values by @sgrebnov in #9114
• Add PollNow interrupt for Ballista executors to reduce task scheduling latency by @phillipleblanc in #9098
• Revert "GCS Data Connector (Alpha) " by @lukekim in #9084
• Fix stack overflow for CDC batching by @krinart in #9115
• fix: update Ballista fork to include executor timeout fix by @phillipleblanc in #9124
• Properly propagate SIGINT/SIGTERM from CLI to runtime by @krinart in #9127
• fix: Use the same vortex dependency as ballista by @peasee in #9123
• release: Bump version to 1.11.0 for stable - https://github.com/spiceai/spiceai/commit/14d09f8e262008df69ded898ed3bebee08471508/
• Cayenne snapshots with shared metadata by @lukekim in #9118
• Improve error handling for URL tables with Azure URLs by @phillipleblanc in #9129
• Add missing Windows build step for spice CLI in build_and_release workflow by @phillipleblanc in #9143
• Fix install-dev to use debug build path for spice binary by @phillipleblanc in #9142
• fix: CLI builds by @peasee in #9145
• Always create initial snapshots (unless bootstrapped) + when no snapshots exist by @krinart in #9119
• fix(cayenne): Fix upsert with pending deletions causing duplicate PKs by @sgrebnov in #9152
• fix(flightrepl): Add chrono-tz feature to flightrepl for timezone formatting by @sgrebnov in #9153
• fix(delta_lake): Preserve container name in ABFSS URLs for Azure Delta Lake tables by @sgrebnov in #9155
• fix: Make CLI system and asset type detection more robust by @peasee in #9148
• fix: Set query set properly on benchmarks telemetry metrics attributes by @peasee in #9162
• fix: Download _models variant - https://github.com/spiceai/spiceai/commit/27f3058d0007595b02c198755c3b22319032ff30/
• fix: Helm chart image tag - https://github.com/spiceai/spiceai/commit/7405c8df0db4ecce0ed6d4a4d424553d604b9036/
• Revert "Remove models variant (now default) & Windows builds (use WSL) " by @lukekim in #9063
• fix(cli): Several CLI fixes from the Go to Rust migration by @lukekim in #9157

Announcing the release of Spice v1.11.0-rc.2! ⭐

v1.11.0-rc.2 is the second release candidate for advanced test of v1.11. It brings Spice Cayenne to Beta status with acceleration snapshots support, a new ScyllaDB Data Connector, upgrades to DataFusion v51, Arrow 57.2, and iceberg-rust v0.8.0. It includes significant improvements to distributed query, caching, and observability.

What's New in v1.11.0-rc.2​

Spice Cayenne Accelerator Reaches Beta​

Spice Cayenne has been promoted to Beta status with acceleration snapshots support and numerous stability improvements.

Improved Reliability:

• Fixed timezone database issues in Docker images that caused acceleration panics
• Resolved FuturesUnordered reentrant drop crashes
• Fixed memory growth issues related to Vortex metrics allocation
• Metadata catalog now properly respects cayenne_file_path location
• Added warnings for unparseable configuration values

Example configuration with snapshots:

datasets:
  - from: s3://my-bucket/data.parquet
    name: my_dataset
    acceleration:
      enabled: true
      engine: cayenne
      mode: file

DataFusion v51 Upgrade​

Apache DataFusion has been upgraded to v51, bringing significant performance improvements, new SQL features, and enhanced observability.

Performance Improvements:

• Faster CASE Expression Evaluation: Expressions now short-circuit earlier, reuse partial results, and avoid unnecessary scattering, speeding up common ETL patterns
• Better Defaults for Remote Parquet Reads: DataFusion now fetches the last 512KB of Parquet files by default, typically avoiding 2 I/O requests per file
• Faster Parquet Metadata Parsing: Leverages Arrow 57's new thrift metadata parser for up to 4x faster metadata parsing

New SQL Features:

• SQL Pipe Operators: Support for |> syntax for inline transforms
• DESCRIBE <query>: Returns the schema of any query without executing it
• Named Arguments in SQL Functions: PostgreSQL-style param => value syntax for scalar, aggregate, and window functions
• Decimal32/Decimal64 Support: New Arrow types supported including aggregations like SUM, AVG, and MIN/MAX

Example pipe operator:

SELECT * FROM t
|> WHERE a > 10
|> ORDER BY b
|> LIMIT 5;

Improved Observability:

• Improved EXPLAIN ANALYZE Metrics: New metrics including output_bytes, selectivity for filters, reduction_factor for aggregates, and detailed timing breakdowns

Arrow 57.2 Upgrade​

Spice has been upgraded to Apache Arrow Rust 57.2.0, bringing major performance improvements and new capabilities.

Key Features:

• 4x Faster Parquet Metadata Parsing: A rewritten thrift metadata parser delivers up to 4x faster metadata parsing, especially beneficial for low-latency use cases and files with large amounts of metadata
• Parquet Variant Support: Experimental support for reading and writing the new Parquet Variant type for semi-structured data, including shredded variant values
• Parquet Geometry Support: Read and write support for Parquet Geometry types (GEOMETRY and GEOGRAPHY) with GeospatialStatistics
• New arrow-avro Crate: Efficient conversion between Apache Avro and Arrow RecordBatches with projection pushdown and vectorized execution support

iceberg-rust v0.8.0 Upgrade​

Spice has been upgraded to iceberg-rust v0.8.0, bringing improved Iceberg table support.

Key Features:

• V3 Metadata Support: Full support for Iceberg V3 table metadata format
• INSERT INTO Partitioned Tables: DataFusion integration now supports inserting data into partitioned Iceberg tables
• Improved Delete File Handling: Better support for position and equality delete files, including shared delete file loading and caching
• SQL Catalog Updates: Implement update_table and register_table for SQL catalog
• S3 Tables Catalog: Implement update_table for S3 Tables catalog
• Enhanced Arrow Integration: Convert Arrow schema to Iceberg schema with auto-assigned field IDs, _file column support, and Date32 type support

Acceleration Snapshots​

Acceleration snapshots enable point-in-time recovery and data versioning for accelerated datasets. Snapshots capture the state of accelerated data at specific points, allowing for fast bootstrap recovery and rollback capabilities.

Key Feature Improvements in v1.11:

• Flexible Triggers: Configure when snapshots are created based on time intervals or stream batch counts
• Automatic Compaction: Reduce storage overhead by compacting older snapshots (DuckDB only)
• Bootstrap Integration: Snapshots can reset cache expiry on load for seamless recovery (DuckDB with Caching refresh mode)
• Smart Creation Policies: Only create snapshots when data has actually changed

Example configuration:

datasets:
  - from: s3://my-bucket/data.parquet
    name: my_dataset
    acceleration:
      enabled: true
      engine: cayenne
      mode: file
      snapshots: enabled
      snapshots_trigger: time_interval
      snapshots_trigger_threshold: 1h
      snapshots_creation_policy: on_changed

Snapshots API and CLI: New API endpoints and CLI commands for managing snapshots programmatically. List, create, and restore snapshots directly from the command line or via HTTP.

For more details, refer to the Acceleration Snapshots Documentation.

ScyllaDB Data Connector​

A new data connector for ScyllaDB, the high-performance NoSQL database compatible with Apache Cassandra. Query ScyllaDB tables directly or accelerate them for faster analytics.

Example configuration:

datasets:
  - from: scylladb:my_keyspace.my_table
    name: scylla_data
    acceleration:
      enabled: true
      engine: duckdb

For more details, refer to the ScyllaDB Data Connector Documentation.

Distributed Query Improvements​

mTLS Verification: Cluster communication between scheduler and executors now supports mutual TLS verification for enhanced security.

Credential Propagation: Azure and GCS credentials are now automatically propagated to executors in cluster mode, enabling access to cloud storage across the distributed query cluster.

Improved Resilience:

• Exponential backoff for scheduler disconnection recovery
• Increased gRPC message size limit from 16MB to 100MB for large query plans
• HTTP health endpoint for cluster executors
• Automatic executor role inference when --scheduler-address is provided

For more details, refer to the Distributed Query Documentation.

Caching Acceleration Mode Improvements​

The Caching Acceleration Mode introduced in v1.10.0 has received significant performance optimizations and reliability fixes in this release.

Performance Optimizations:

• Non-blocking Cache Writes: Cache misses no longer block query responses. Data is written to the cache asynchronously after the query returns, reducing query latency for cache miss scenarios.
• Batch Cache Writes: Multiple cache entries are now written in batches rather than individually, significantly improving write throughput for high-volume cache operations.

Reliability Fixes:

• Correct SWR Refresh Behavior: The stale-while-revalidate (SWR) pattern now correctly refreshes only the specific entries that were accessed instead of refreshing all stale rows in the dataset. This prevents unnecessary source queries and reduces load on upstream data sources.
• Deduplicated Refresh Requests: Fixed an issue where JSON array responses could trigger multiple redundant refresh operations. Refresh requests are now properly deduplicated.
• Fixed Cache Hit Detection: Resolved an issue where queries that didn't include fetched_at in their projection would always result in cache misses, even when cached data was available.
• Unfiltered Query Optimization: SELECT * queries without filters now return cached data directly without unnecessary filtering overhead.

For more details, refer to the Caching Acceleration Mode Documentation.

DynamoDB Connector Enhancements​

• Added JSON nesting for DynamoDB Streams
• Proper batch deletion handling

URL Tables​

Query data sources directly via URL in SQL without prior dataset registration. Supports S3, Azure Blob Storage, and HTTP/HTTPS URLs with automatic format detection and partition inference.

Supported Patterns:

• Single files: SELECT * FROM 's3://bucket/data.parquet'
• Directories/prefixes: SELECT * FROM 's3://bucket/data/'
• Glob patterns: SELECT * FROM 's3://bucket/year=*/month=*/data.parquet'

Key Features:

• Automatic file format detection (Parquet, CSV, JSON, etc.)
• Hive-style partition inference with filter pushdown
• Schema inference from files
• Works with both SQL and DataFrame APIs

Example with hive partitioning:

-- Partitions are automatically inferred from paths
SELECT * FROM 's3://bucket/data/' WHERE year = '2024' AND month = '01'

Enable via spicepod.yml:

runtime:
  params:
    url_tables: enabled

Cluster Mode Async Query APIs (experimental)​

New asynchronous query APIs for long-running queries in cluster mode:

• /v1/queries endpoint: Submit queries and retrieve results asynchronously
• Arrow Flight async support: Non-blocking query execution via Arrow Flight protocol

Observability Improvements​

Enhanced Dashboards: Updated Grafana and Datadog example dashboards with:

• Snapshot monitoring widgets
• Improved accelerated datasets section
• Renamed ingestion lag charts for clarity

Additional Histogram Buckets: Added more buckets to histogram metrics for better latency distribution visibility.

For more details, refer to the Monitoring Documentation.

Additional Improvements​

• Model Listing: New functionality to list available models across multiple AI providers
• DuckDB Partitioned Tables: Primary key constraints now supported in partitioned DuckDB table mode
• Post-refresh Sorting: New on_refresh_sort_columns parameter for DuckDB enables data ordering after writes
• Improved Install Scripts: Removed jq dependency and improved cross-platform compatibility
• Better Error Messages: Improved error messaging for bucket UDF arguments and deprecated OpenAI parameters

Contributors​

• @ewgenius
• @Jeadie
• @krinart
• @lukekim
• @peasee
• @phillipleblanc
• @sgrebnov

Breaking Changes​

No breaking changes.

Cookbook Updates​

New ScyllaDB Data Connector Recipe: New recipe demonstrating how to use ScyllaDB Data Connector. See ScyllaDB Data Connector Recipe for details.

New SMB Data Connector Recipe: New recipe demonstrating how to use ScyllaDB Data Connector. See SMB Data Connector Recipe for details.

The Spice Cookbook includes 86 recipes to help you get started with Spice quickly and easily.

Upgrading​

To upgrade to v1.11.0-rc.2, use one of the following methods:

CLI:

spice upgrade

Homebrew:

brew upgrade spiceai/spiceai/spice

Docker:

Pull the spiceai/spiceai:v1.11.0-rc.2 image:

docker pull spiceai/spiceai:v1.11.0-rc.2

For available tags, see DockerHub.

Helm:

helm repo update
helm upgrade spiceai spiceai/spiceai

AWS Marketplace:

Spice is available in the AWS Marketplace.

Dependencies​

• DataFusion: Upgraded to v51 (release notes)
• Arrow: Upgraded to v57 (release notes)
• iceberg-rust: Upgraded to v0.8.0 (release notes)

Changelog​

• Add timezone database to Docker image to fix Cayenne acceleration panic by @sgrebnov in #8799
• Upgrade dependencies by @phillipleblanc in #8801
• Fix table_allowlist for table sampling and NSQL by @Jeadie in #8789
• Cayenne primary key on-conflict handling by @lukekim in #8788
• fix: Update benchmark snapshots by @app/github-actions in #8773
• fix: correctly identify deprecated openai_* parameters by @phillipleblanc in #8809
• fix: Update benchmark snapshots by @app/github-actions in #8812
• Use workspace version for cayenne crate by @phillipleblanc in #8811
• Don't CAST strings which breaks push down optimizer by @lukekim in #8810
• fix: Update benchmark snapshots by @app/github-actions in #8815
• Update async-openai to latest revision 4dcd633aad6f - brings fix for openai compatible model providers by @ewgenius in #8816
• Add auth/iam_role_source to DynamoDB connector by @krinart in #8808
• DynamoDB fixes: JSON nesting for Streams, proper batch deletions by @krinart in #8821
• Rough roadmap for 2026-2027 by @lukekim in #8805
• Release notes for v1.11.0-rc1 by @ewgenius in #8786
• Make S3V integration tests prepare_for_aws_tests more robust by @sgrebnov in #8820
• Bump rsa from 0.9.9 to 0.9.10 in the cargo group across 1 directory by @app/dependabot in #8819
• Add timezone database to Release and CUDA Docker images to fix Cayene panic by @sgrebnov in #8832
• fix: UnionProjectionPushdownOptimizer - Schema change during transform_down breaks parent nodes by @phillipleblanc in #8831
• Update grafana/datadog example dashboards by @krinart in #8833
• Add Dev bird bench as text-to-sql queryset in CI. by @Jeadie in #8753
• Update testoperator scheduler to use release/1.11 branch by @ewgenius in #8829
• Spice Cayenne fixes and test spicepods for Beta & RC by @lukekim in #8787
• testoperator dispatch all bird-bench database variants by @Jeadie in #8835
• feat: Improve column statistics handling with safe access and defaults by @phillipleblanc in #8836
• cluster: mTLS verification by @phillipleblanc in #8837
• fix: 8770: Unsupported ScalarFunctionExpr in ORDER BY by @lukekim in #8838
• Workflow tweaks by @lukekim in #8845
• Cayenne: metadata catalog should respect cayenne_file_path location by @sgrebnov in #8844
• Expand Cayenne feature coverage by @lukekim in #8848
• docs: HA distributed query decisions by @phillipleblanc in #8817
• fix(optimizer): Fix correctness issues in UnionProjectionPushdownOptimizer by @phillipleblanc in #8851
• Pin reqwest to 0.12.24 to fix HuggingFace embedding model download by @ewgenius in #8853
• Fix builds and pin to Ubuntu 22.04 by @lukekim in #8856
• Revert "Fix builds and pin to Ubuntu 22.04" by @lukekim in #8861
• Ensure setup Rust is run by @lukekim in #8862
• fix: Ubuntu 24.04+ renamed libaio1 to libaio1t64 by @lukekim in #8865
• Upgrade to Pulls with Spice v2 by @lukekim in #8866
• Add limit and configuration name to 'testoperator run text-to-sql' by @Jeadie in #8839
• PR check and test optimization by @lukekim in #8868
• Upgrade S3 Vectors SDK and improve test robustness by @lukekim in #8867
• [Testoperator] Query level and improved aggregate level for NSQL by @Jeadie in #8840
• Add docker build for private branches for ghcr.io/spiceai/spiceai-dev by @phillipleblanc in #8873
• Expand the data acceleration round-trip test coverage by @lukekim in #8855
• fix: Provide a better error for improper bucket UDF arguments by @peasee in #8849
• ScyllaDB Data Connector by @lukekim in #8827
• Use tokio-rusqlite for Spice Cayenne SQLite by @lukekim in #8857
• Cayenne: fix FuturesUnordered reentrant drop crash by @sgrebnov in #8863
• Bump github/codeql-action from 4.31.9 to 4.31.10 by @app/dependabot in #8884
• Bump golang.org/x/sys from 0.39.0 to 0.40.0 by @app/dependabot in #8881
• Bump github.com/spiceai/gospice/v8 from 8.0.0 to 8.0.1 by @app/dependabot in #8883
• Bump roaring from 0.11.2 to 0.11.3 by @app/dependabot in #8885
• Bump golang.org/x/mod from 0.31.0 to 0.32.0 by @app/dependabot in #8882
• Bump aws-sdk-s3 from 1.115.0 to 1.119.0 by @app/dependabot in #8887
• Bump libc from 0.2.177 to 0.2.180 by @app/dependabot in #8886
• Bump tokio-util from 0.7.17 to 0.7.18 by @app/dependabot in #8889
• Bump governor from 0.10.2 to 0.10.4 by @app/dependabot in #8888
• fix: flaky test test_concurrent_partition_creation by @phillipleblanc in #8898
• Update Cayenne snapshots for TPC-DS by @lukekim in #8890
• Add more buckets to histogram metrics by @krinart in #8850
• feat: Add HTTP health endpoint for cluster executors by @phillipleblanc in #8899
• feat: Implement model listing functionality for multiple providers by @lukekim in #8901
• feat: Initial HA schedulers distributed query implementation by @phillipleblanc in #8852
• fix: infer executor role from --scheduler-address when --role is omitted by @phillipleblanc in #8903
• Improve install scripts and remove jq dependency by @lukekim in #8847
• Benchmarks: sort PartitionedUnionExec children for deterministic snapshot comparison by @sgrebnov in #8877
• Cayenne: share VortexFileCache across partitions via CayenneContext by @sgrebnov in #8880
• Update ballista to add exponential backoff for scheduler disconnection by @phillipleblanc in #8905
• Configurably add BirdBench evidence to testoperator text-to-SQL. by @Jeadie in #8904
• Helm: Allow command override via values.yaml by @sgrebnov in #8906
• Fix distributed query gRPC message size limit (16MB -> 100MB) by @phillipleblanc in #8900
• OS specific setup actions by @lukekim in #8909
• Cayenne should warn if unable to parse configuration value by @sgrebnov in #8907
• Add snapshots widgets to example dashboard by @krinart in #8910
• Add quality criteria for the features by @krinart in #8897
• Improve Accelerated Datasets section for Grafana/Datadog dashboards by @krinart in #8915
• Use HTTP traceparent in NSQL to support concurrency in 'testoperator run text-to-SQL' by @Jeadie in #8912
• Remove setup for cc from integration_models.yml by @Jeadie in #8917
• Propagate Azure and GCS credentials to executors in cluster mode by @phillipleblanc in #8918
• Cayenne: fix memory growth due to vortex metrics allocation by @sgrebnov in #8908
• fix(caching): Deduplicate refresh requests for JSON array responses by @sgrebnov in #8921
• fix(caching): Return cached data directly for unfiltered queries (SELECT *) by @sgrebnov in #8919
• Correct MinIO path syntax for spiced download by @Jeadie in #8916
• Acceleration snapshots compaction + Improved Snapshots UX by @krinart in #8858
• Change base image from bookworm-slim to trixie-slim by @Jeadie in #8923
• Add testoperator run text-to-sql metrics from LogicalPlan by @Jeadie in #8895
• Fix spicepod dependencies in testoperator by @Jeadie in #8875
• Update copilot instructions for data correctness by @lukekim in #8922
• Add BootstrapStatus + Snapshot bootstrapping parallelization by @krinart in #8926
• fix: add missing feature-gate for AWS Secrets Manager error variant by @phillipleblanc in #8928
• refactor: make ConnectorParams fields public for external connectors by @phillipleblanc in #8929
• fix(caching): SWR refreshes only accessed entry instead of all stale rows by @sgrebnov in #8931
• Cayenne: include cayenne_metadata_dir to known params by @sgrebnov in #8933
• Rename Ingestion Lag chart in example dashboards by @krinart in #8932
• fix(caching): Fix HTTP caching always MISS when projection excludes fetched_at by @sgrebnov in #8930
• Reset expiry after snapshot bootstraping for Caching by @krinart in #8925
• Set use_ssl=false for sccache by @lukekim in #8945
• Hash indexing for Arrow Acceleration by @lukekim in #8924
• [Cayenne] Acceleration snapshots support by @lukekim in #7973
• perf(caching): Non-blocking cache writes on cache miss by @sgrebnov in #8948
• Update NSQL models by @lukekim in #8951
• Hash Index Key verification by @lukekim in #8949
• Add snapshots_creation_policy param by @krinart in #8954
• Remove candle & cudarc from non-models build by @lukekim in #8955
• Acceleration Snapshots API and CLI by @lukekim in #8934
• Ignore test for data_components arrow::indexed::test_primary_key_value_matches_batch by @Jeadie in #8962
• fix: Update benchmark snapshots by @app/github-actions in #8965
• Hash Index secondary index support by @lukekim in #8958
• fix: Support primary key constraints in partitioned DuckDB tables mode by @sgrebnov in #8966
• perf(caching): Batch cache writes by @sgrebnov in #8959
• CI perf optimizations by @lukekim in #8968
• Fix Makefile linting by @Jeadie in #8970
• Fixes in testoperator run text-to-sql. by @Jeadie in #8927
• implement Chat::as_sql for xAI anthropic by @Jeadie in #8957
• Fix duckdb_file_path in search integration test by @Jeadie in #8972
• fix: Update benchmark snapshots by @app/github-actions in #8971
• Maintenance updates to Anthropic API by @Jeadie in #8956
• Add CacheBackend Trait, implement pingora-lru, and add throughput tests by @lukekim in #8080
• fix: Update benchmark snapshots by @app/github-actions in #8974
• fix: Update benchmark snapshots by @app/github-actions in #8975
• Make accelerator shutdown more robust by @lukekim in #8969
• feat(duckdb): Add on_refresh_sort_columns for post-write data ordering (initial version) by @sgrebnov in #8964
• Proper handling for initial snapshot by @krinart in #8911
• fix: Remove --no-default-features from cargo-hack command in features workflow by @phillipleblanc in #8977
• build(deps): bump actions/cache from 5.0.1 to 5.0.2 by @app/dependabot in #8983
• build(deps): bump actions/checkout from 4 to 6 by @app/dependabot in #8982
• build(deps): bump actions/setup-go from 6.1.0 to 6.2.0 by @app/dependabot in #8984
• build(deps): bump github.com/olekukonko/tablewriter from 1.1.2 to 1.1.3 by @app/dependabot in #8979
• build(deps): bump github.com/klauspost/compress from 1.18.2 to 1.18.3 by @app/dependabot in #8980
• Add /v1/queries and Arrow Flight async APIs by @lukekim in #8946
• build(deps): bump Vampire/setup-wsl from 5 to 6 by @app/dependabot in #8981
• fix: Update Search integration test snapshots by @app/github-actions in #8973
• build(deps): bump insta from 1.46.0 to 1.46.1 by @app/dependabot in #8988
• build(deps): bump schemars from 1.1.0 to 1.2.0 by @app/dependabot in #8985
• fix: Update benchmark snapshots by @app/github-actions in #8978
• fix: Data correctness edge cases by @lukekim in #8953
• Correct MinIO path syntax for spiced download (Part 2) by @Jeadie in #8995
• Make .spice/data in search integration tests by @Jeadie in #8992
• fix: Hash index composite keys null values by @lukekim in #9001
• Update Cayenne status to Beta by @lukekim in #9002
• fix: Disable TPC-DS result validation (not yet supported) by @sgrebnov in #9004
• feat: Upgrade to DataFusion v51 and dependencies by @lukekim in #8864
• Improvements for snapshots_creation_policy by @krinart in #9003
• fix(ci): restore cached spicepod-validator binary instead of lookup-only by @phillipleblanc in #9007
• Update version by @krinart in #9010