Lance
GitHub Get started
Docs /Getting Started /LanceDB

Getting Started with LanceDB

Lance is an open table format. You can work with Lance tables from many engines, and this quickstart covers:

  • LanceDB (this page) is a retrieval database built on Lance. It runs embedded in your application or as a distributed service.
  • Apache Spark runs SQL and distributed data processing on Lance tables.
  • Format SDKs (PyLance, Rust, Java) provide low-level control over Lance datasets when building custom engines or pipelines.

All of these tools read and write the same format, so a table you create with one of them can be opened with the others.

In this guide, you'll create a small product catalog with LanceDB, filter it, search it by vector similarity and by keyword, modify it, and look back at earlier versions.

Install LanceDB

This guide displays results as pandas DataFrames, so install pandas as well:

pip install lancedb pandas

Connect to a Database

This guide uses LanceDB in embedded mode, where a database is a directory of Lance tables, on local disk or in object storage such as s3://. Connect to a local path; LanceDB creates it if it doesn't exist:

import lancedb

db = lancedb.connect("/tmp/lancedb-quickstart")

The same client connects to a distributed LanceDB deployment when you pass a db:// URI instead of a path.

Create a Table

Create a table from a list of dictionaries. LanceDB infers the schema and stores the vector column as a fixed-size vector:

data = [
    {"id": 1, "name": "red running shoes", "category": "shoes", "price": 89.0, "vector": [1.0, 0.0, 0.0, 0.25]},
    {"id": 2, "name": "blue trail running shoes", "category": "shoes", "price": 120.0, "vector": [0.875, 0.125, 0.0, 0.375]},
    {"id": 3, "name": "leather hiking boots", "category": "shoes", "price": 150.0, "vector": [0.625, 0.0, 0.5, 0.25]},
    {"id": 4, "name": "waterproof rain jacket", "category": "outerwear", "price": 99.0, "vector": [0.0, 1.0, 0.25, 0.0]},
    {"id": 5, "name": "wool winter coat", "category": "outerwear", "price": 210.0, "vector": [0.25, 0.75, 0.625, 0.0]},
]
table = db.create_table("products", data=data)
table.to_pandas()

   id                      name   category  price                      vector
0   1         red running shoes      shoes   89.0       [1.0, 0.0, 0.0, 0.25]
1   2  blue trail running shoes      shoes  120.0  [0.875, 0.125, 0.0, 0.375]
2   3      leather hiking boots      shoes  150.0     [0.625, 0.0, 0.5, 0.25]
3   4    waterproof rain jacket  outerwear   99.0       [0.0, 1.0, 0.25, 0.0]
4   5          wool winter coat  outerwear  210.0    [0.25, 0.75, 0.625, 0.0]

LanceDB also accepts pandas DataFrames, Polars DataFrames, and Arrow tables as input.

Filter Rows

Use SQL expressions to filter rows and choose columns:

table.search().where("price < 100").select(["id", "name", "price"]).to_pandas()

   id                    name  price
0   1       red running shoes   89.0
1   4  waterproof rain jacket   99.0

Search by Vector Similarity

Pass a query vector to search to find the nearest rows. The _distance column holds each row's distance from the query:

query = [0.875, 0.125, 0.0, 0.25]
table.search(query).limit(3).select(["id", "name", "_distance"]).to_pandas()

   id                      name  _distance
0   2  blue trail running shoes     0.0156
1   1         red running shoes     0.0312
2   3      leather hiking boots     0.3281

You can combine vector search with a filter:

(
    table.search(query)
    .where("category = 'outerwear'")
    .limit(1)
    .select(["id", "name", "_distance"])
    .to_pandas()
)

   id              name  _distance
0   5  wool winter coat     1.2344

On a small table like this one, LanceDB compares the query against every vector. For larger tables, create a vector index to keep searches fast.

Search by Keyword

Create a full-text search index on a text column, then search it with keywords. The _score column holds each row's relevance score:

from lancedb.index import FTS

table.create_index("name", config=FTS())
table.search("running", query_type="fts").to_pandas()

   id                      name category  price                      vector  _score
0   1         red running shoes    shoes   89.0       [1.0, 0.0, 0.0, 0.25]  0.8984
1   2  blue trail running shoes    shoes  120.0  [0.875, 0.125, 0.0, 0.375]  0.7942

Add, Update, and Delete Rows

table.add(
    [{"id": 6, "name": "trail socks", "category": "accessories", "price": 15.0, "vector": [0.75, 0.25, 0.25, 0.25]}]
)
table.update(where="id = 1", values={"price": 79.0})
table.delete("category = 'outerwear'")
table.to_pandas()

   id                      name     category  price                      vector
0   2  blue trail running shoes        shoes  120.0  [0.875, 0.125, 0.0, 0.375]
1   3      leather hiking boots        shoes  150.0     [0.625, 0.0, 0.5, 0.25]
2   6               trail socks  accessories   15.0    [0.75, 0.25, 0.25, 0.25]
3   1         red running shoes        shoes   79.0       [1.0, 0.0, 0.0, 0.25]

Time Travel

Every change creates a new version of the table:

table.version

5

Earlier versions stay readable. Check out version 1 to see the table as it was when you created it:

table.checkout(1)
table.to_pandas()

   id                      name   category  price                      vector
0   1         red running shoes      shoes   89.0       [1.0, 0.0, 0.0, 0.25]
1   2  blue trail running shoes      shoes  120.0  [0.875, 0.125, 0.0, 0.375]
2   3      leather hiking boots      shoes  150.0     [0.625, 0.0, 0.5, 0.25]
3   4    waterproof rain jacket  outerwear   99.0       [0.0, 1.0, 0.25, 0.0]
4   5          wool winter coat  outerwear  210.0    [0.25, 0.75, 0.625, 0.0]

Return to the latest version to continue making changes:

table.checkout_latest()
table.count_rows()

4

Next Steps

  • Read the LanceDB documentation for embeddings, hybrid search, reranking, distributed deployments, and the TypeScript and Rust SDKs.
  • See the Spark quickstart to process Lance tables at scale with SQL.
  • See the format SDK quickstarts for PyLance, Rust, and Java to work directly with the Lance format.
← Previous
Home