Zone Map Index¶

Zone maps are a columnar database technique for predicate pushdown and scan pruning. They break data into fixed-size chunks called "zones" and maintain summary statistics (min, max, null count) for each zone, enabling efficient filtering by eliminating zones that cannot contain matching values.

Zone maps are "inexact" filters - they can definitively exclude zones but may include false positives that require rechecking.

In addition, since finding NULLs is a common query pattern, the index also maintains a bitmap of null rows which allows it to return exact results for IS NULL queries.

Index Details¶

message ZoneMapIndexDetails {
  // Number of rows per zone. Optional for backwards compatibility: absent on
  // datasets written before this field was added. When absent, no seed writer
  // is created for the index.
  optional uint64 rows_per_zone = 1;
  // Whether seed-based incremental updates are enabled for this index.
  // On-disk semantics: absent means seeds are disabled (old datasets written
  // before this field was added). Present false means explicitly disabled.
  // Present true means seeds are enabled: the index will embed per-fragment
  // seed buffers in data files and harvest them during incremental updates
  // to skip full column scans.
  // Creation-time default: index creation code sets this to true for
  // variable-length types (strings, binary) and fixed-width types wider than
  // 8 bytes, and to false for narrow fixed-width types (e.g. Int64, Float64).
  optional bool use_seeds = 2;

}

Storage Layout¶

The zone map index stores zone statistics in a single file:

1. zonemap.lance - Zone statistics for query pruning

Zone Statistics File Schema¶

Schema Metadata¶

Global Buffers¶

Accelerated Queries¶

The zone map index provides inexact results for the following query types (nullability queries return exact results):