Lance
Docs /Integrations /DuckDB /REST Namespace

Connecting to a Lance REST Namespace

This guide explains how to connect DuckDB to a Lance REST Namespace server (e.g. LanceDB Cloud or Enterprise) using the lance-duckdb extension.

Overview

A Lance REST Namespace provides a centralized catalog for managing Lance tables. The REST Namespace API allows:

  • Listing tables in a namespace
  • Creating and dropping tables
  • Describing tables (getting location and storage credentials)
  • Querying tables through the query_table API
  • Credential vending for accessing underlying storage (S3, GCS, Azure, etc.)

Prerequisites

  1. A Lance REST Namespace server endpoint available
  2. The lance-duckdb extension built and available
  3. Authentication credentials (API key, bearer token, etc.)

ATTACH Syntax

ATTACH '<namespace_id>' AS <alias> (
    TYPE LANCE,
    ENDPOINT '<server_url>',
    HEADER '<header1>=<value1>;<header2>=<value2>'
);

Parameters

Parameter Description Required
<namespace_id> The namespace identifier to connect to Yes
<alias> Local alias for the attached database Yes
ENDPOINT URL of the REST Namespace server Yes
HEADER Custom HTTP headers (semicolon-separated key=value pairs) No
DELIMITER Namespace delimiter (default: $) No
BEARER_TOKEN Bearer token for authentication No
API_KEY API key for authentication No

Table names and permissions

REST attachments store the connection configuration and resolve tables lazily. A point lookup describes the table directly, so it can succeed with permission to describe and read that table even when namespace listing is denied. SHOW TABLES and CREATE conflict checks require listing permission. Endpoint, authentication, and namespace errors are reported when the relevant operation runs. A server-reported TableNotFound becomes DuckDB's normal missing-table result; other failures remain errors.

SQL table names are leaf names within the attached namespace. For example, lance_ns.main.orders under ATTACH 'default' addresses the remote ID default$orders. Reads and writes use that complete ID. With the default $ delimiter, lance_ns.main."orders.2026" addresses the single table component orders.2026. A table name containing the active delimiter is rejected because the REST protocol cannot distinguish it from a namespace boundary. Composed identifiers must also round-trip through the configured delimiter. For example, namespace a: plus child b cannot be represented with delimiter ::, because a:::b decodes as a and :b; such operations fail before remote mutation. Use the delimiter itself as the namespace ID to attach the root namespace.

Table enumeration accepts either full IDs with the attached namespace prefix or relative leaf names. Deeper descendants and IDs from other namespaces are excluded from this schema.

Namespaces and schemas

The attached namespace is exposed as DuckDB main. Its direct child namespaces are exposed as schemas in the same catalog:

ATTACH 'default' AS ns (TYPE LANCE, ENDPOINT 'http://localhost:10024');
SELECT * FROM ns.main.orders;
SELECT * FROM ns.analytics.orders;
CREATE SCHEMA ns.reporting;
CREATE TABLE ns.reporting.daily AS SELECT 42 AS total;
DROP TABLE ns.reporting.daily;
DROP SCHEMA ns.reporting;

Here ns.analytics.orders addresses default$analytics$orders. Every table operation uses the namespace bound to its schema. Deeper namespace trees can be accessed through another ATTACH; nested schema CREATE/DROP is unsupported. Names containing dots or newlines remain single quoted SQL identifiers when the configured REST delimiter permits them. Empty names, reserved schema names (main, information_schema, pg_catalog), delimiter-bearing names, and case-insensitive collisions in child discovery are rejected.

Child discovery is lazy and collects every page before publishing a complete snapshot. Each attachment keeps that snapshot; use DETACH/ATTACH to observe external namespace changes. Successful schema CREATE/DROP updates the local snapshot. Table discovery and metadata freshness retain their existing behavior. Direct main table lookup does not require child discovery. DuckDB system catalog queries, including SHOW TABLES, enumerate schemas before filtering and therefore may require namespace listing permission. If the backend explicitly does not support namespace listing, system catalog queries expose known local entries; named child discovery still reports the unsupported operation. Authentication, transport, and malformed-response errors are not hidden.

Schema DDL and REST table DDL require autocommit. CREATE OR REPLACE SCHEMA is unsupported. DROP SCHEMA ... RESTRICT and ... CASCADE use the backend's namespace operation after DuckDB checks local dependencies. A remote failure rolls back DuckDB's local schema/view changes; it does not provide a distributed rollback guarantee for an uncertain remote outcome. Cross-schema view dependency tracking follows DuckDB's enable_view_dependencies setting.

Legacy directory attachments have manifest mode disabled and expose main only; they do not support child schema CREATE.

Custom Headers

The HEADER option allows passing custom HTTP headers to the REST Namespace server. Multiple headers can be specified using semicolon separation:

HEADER 'x-lancedb-database=my_db;x-api-key=sk_xxx;x-custom-header=value'

Example: Connecting to a REST Namespace Server

1. Connect from DuckDB

-- Load the Lance extension
LOAD 'lance.duckdb_extension';

-- Attach to the REST Namespace
ATTACH 'ns1' AS lance_ns (
    TYPE LANCE,
    ENDPOINT 'http://localhost:10024',
    HEADER 'x-lancedb-database=lance_ns;x-api-key=sk_localtest'
);

-- Switch to the attached database
USE lance_ns;

2. List Tables

SHOW TABLES;

Output:

┌─────────┐
│  name   │
│ varchar │
├─────────┤
│ 0 rows  │
└─────────┘

3. Create a Table

CREATE TABLE users (
    id INTEGER,
    name VARCHAR,
    email VARCHAR
);

4. Insert Data

INSERT INTO users VALUES
    (1, 'Alice', 'alice@example.com'),
    (2, 'Bob', 'bob@example.com'),
    (3, 'Charlie', 'charlie@example.com');

5. Query Data

For tables attached through a REST Namespace, SELECT is executed through the Lance Namespace query_table API. DuckDB pushes the required projection, supported WHERE predicates, and LIMIT/OFFSET pairs into the request and reads the Arrow IPC response. Reads therefore do not require DuckDB to open the underlying object-store location directly. Unsupported predicates remain in DuckDB, and an OFFSET without a LIMIT is also evaluated by DuckDB.

Results are matched by column name and decoded using the Arrow types in the response, including dictionary and large-offset encodings. Missing or duplicate columns and incompatible SQL types produce an error. Namespace searches also preserve catalog types such as UUID. REST descriptions represent timestamps in microseconds without a timezone; response timestamps preserve their epoch value when converted to that type, and sub-microsecond values that cannot be represented exactly produce an error.

If the namespace server does not implement query_table, disable the lance_namespace_query_table setting to scan by opening the underlying dataset directly (table locations are still resolved through the namespace, and reads then require direct storage access):

SET lance_namespace_query_table = false;

Direct storage access prefers non-empty storage_options returned by the namespace's describe_table response. If that response omits the field or returns an empty map, the extension looks up a matching DuckDB TYPE LANCE secret by the resolved dataset location. Replacing that secret changes the dataset cache identity, so subsequent statements open with the new credentials.

-- Select all rows
SELECT * FROM users;

Output:

┌───────┬─────────┬─────────────────────┐
│  id   │  name   │        email        │
│ int32 │ varchar │       varchar       │
├───────┼─────────┼─────────────────────┤
│     1 │ Alice   │ alice@example.com   │
│     2 │ Bob     │ bob@example.com     │
│     3 │ Charlie │ charlie@example.com │
└───────┴─────────┴─────────────────────┘

-- Select with filter
SELECT * FROM users WHERE id > 1;

Output:

┌───────┬─────────┬─────────────────────┐
│  id   │  name   │        email        │
│ int32 │ varchar │       varchar       │
├───────┼─────────┼─────────────────────┤
│     2 │ Bob     │ bob@example.com     │
│     3 │ Charlie │ charlie@example.com │
└───────┴─────────┴─────────────────────┘

-- Aggregation
SELECT COUNT(*) as total_users FROM users;

Output:

┌─────────────┐
│ total_users │
│    int64    │
├─────────────┤
│           3 │
└─────────────┘

6. Using Fully Qualified Names

You can also use fully qualified table names without switching databases:

-- Create table with fully qualified name
CREATE TABLE lance_ns.main.my_table (col1 INTEGER, col2 VARCHAR);

-- Query with fully qualified name
SELECT * FROM lance_ns.main.my_table;