Connecting to a Lance REST Namespace¶
This guide explains how to connect DuckDB to a Lance REST Namespace server (e.g. LanceDB Cloud or Enterprise) using the lance-duckdb extension.
Overview¶
A Lance REST Namespace provides a centralized catalog for managing Lance tables. The REST Namespace API allows:
- Listing tables in a namespace
- Creating and dropping tables
- Describing tables (getting location and storage credentials)
- Querying tables through the
query_tableAPI - Credential vending for accessing underlying storage (S3, GCS, Azure, etc.)
Prerequisites¶
- A Lance REST Namespace server endpoint available
- The lance-duckdb extension built and available
- Authentication credentials (API key, bearer token, etc.)
ATTACH Syntax¶
ATTACH '<namespace_id>' AS <alias> (
TYPE LANCE,
ENDPOINT '<server_url>',
HEADER '<header1>=<value1>;<header2>=<value2>'
);
Parameters¶
| Parameter | Description | Required |
|---|---|---|
<namespace_id> |
The namespace identifier to connect to | Yes |
<alias> |
Local alias for the attached database | Yes |
ENDPOINT |
URL of the REST Namespace server | Yes |
HEADER |
Custom HTTP headers (semicolon-separated key=value pairs) | No |
DELIMITER |
Namespace delimiter (default: $) |
No |
BEARER_TOKEN |
Bearer token for authentication | No |
API_KEY |
API key for authentication | No |
Table names and permissions¶
REST attachments store the connection configuration and resolve tables lazily.
A point lookup describes the table directly, so it can succeed with permission
to describe and read that table even when namespace listing is denied.
SHOW TABLES and CREATE conflict checks require listing permission.
Endpoint, authentication, and namespace errors are reported when the relevant
operation runs. A server-reported TableNotFound becomes DuckDB's normal
missing-table result; other failures remain errors.
SQL table names are leaf names within the attached namespace. For example,
lance_ns.main.orders under ATTACH 'default' addresses the remote ID
default$orders. Reads and writes use that complete ID. With the default $
delimiter, lance_ns.main."orders.2026" addresses the single table component
orders.2026. A table name containing the active delimiter is rejected because
the REST protocol cannot distinguish it from a namespace boundary. Composed
identifiers must also round-trip through the configured delimiter. For example,
namespace a: plus child b cannot be represented with delimiter ::, because
a:::b decodes as a and :b; such operations fail before remote mutation.
Use the delimiter itself as the namespace ID to attach the root namespace.
Table enumeration accepts either full IDs with the attached namespace prefix or relative leaf names. Deeper descendants and IDs from other namespaces are excluded from this schema.
Namespaces and schemas¶
The attached namespace is exposed as DuckDB main. Its direct child namespaces
are exposed as schemas in the same catalog:
ATTACH 'default' AS ns (TYPE LANCE, ENDPOINT 'http://localhost:10024');
SELECT * FROM ns.main.orders;
SELECT * FROM ns.analytics.orders;
CREATE SCHEMA ns.reporting;
CREATE TABLE ns.reporting.daily AS SELECT 42 AS total;
DROP TABLE ns.reporting.daily;
DROP SCHEMA ns.reporting;
Here ns.analytics.orders addresses default$analytics$orders. Every table
operation uses the namespace bound to its schema. Deeper namespace trees can
be accessed through another ATTACH; nested schema CREATE/DROP is unsupported.
Names containing dots or newlines remain single quoted SQL identifiers when
the configured REST delimiter permits them. Empty names, reserved schema names
(main, information_schema, pg_catalog), delimiter-bearing names, and
case-insensitive collisions in child discovery are rejected.
Child discovery is lazy and collects every page before publishing a complete
snapshot. Each attachment keeps that snapshot; use DETACH/ATTACH to observe
external namespace changes. Successful schema CREATE/DROP updates the local
snapshot. Table discovery and metadata freshness retain their existing behavior.
Direct main table lookup does not require child discovery. DuckDB system
catalog queries, including SHOW TABLES, enumerate schemas before filtering and
therefore may require namespace listing permission. If the backend explicitly
does not support namespace listing, system catalog queries expose known local
entries; named child discovery still reports the unsupported operation.
Authentication, transport, and malformed-response errors are not hidden.
Schema DDL and REST table DDL require autocommit. CREATE OR REPLACE SCHEMA
is unsupported. DROP SCHEMA ... RESTRICT and ... CASCADE use the backend's
namespace operation after DuckDB checks local dependencies. A remote failure
rolls back DuckDB's local schema/view changes; it does not provide a distributed
rollback guarantee for an uncertain remote outcome. Cross-schema view dependency
tracking follows DuckDB's enable_view_dependencies setting.
Legacy directory attachments have manifest mode disabled and expose main
only; they do not support child schema CREATE.
Custom Headers¶
The HEADER option allows passing custom HTTP headers to the REST Namespace server. Multiple headers can be specified using semicolon separation:
Example: Connecting to a REST Namespace Server¶
1. Connect from DuckDB¶
-- Load the Lance extension
LOAD 'lance.duckdb_extension';
-- Attach to the REST Namespace
ATTACH 'ns1' AS lance_ns (
TYPE LANCE,
ENDPOINT 'http://localhost:10024',
HEADER 'x-lancedb-database=lance_ns;x-api-key=sk_localtest'
);
-- Switch to the attached database
USE lance_ns;
2. List Tables¶
Output:
3. Create a Table¶
4. Insert Data¶
INSERT INTO users VALUES
(1, 'Alice', 'alice@example.com'),
(2, 'Bob', 'bob@example.com'),
(3, 'Charlie', 'charlie@example.com');
5. Query Data¶
For tables attached through a REST Namespace, SELECT is executed through the
Lance Namespace query_table API. DuckDB pushes the required projection,
supported WHERE predicates, and LIMIT/OFFSET pairs into the request and
reads the Arrow IPC response. Reads therefore do not require DuckDB to open the
underlying object-store location directly. Unsupported predicates remain in
DuckDB, and an OFFSET without a LIMIT is also evaluated by DuckDB.
Results are matched by column name and decoded using the Arrow types in the response, including dictionary and large-offset encodings. Missing or duplicate columns and incompatible SQL types produce an error. Namespace searches also preserve catalog types such as UUID. REST descriptions represent timestamps in microseconds without a timezone; response timestamps preserve their epoch value when converted to that type, and sub-microsecond values that cannot be represented exactly produce an error.
If the namespace server does not implement query_table, disable the
lance_namespace_query_table setting to scan by opening the underlying
dataset directly (table locations are still resolved through the namespace,
and reads then require direct storage access):
Direct storage access prefers non-empty storage_options returned by the
namespace's describe_table response. If that response omits the field or
returns an empty map, the extension looks up a matching DuckDB TYPE LANCE
secret by the resolved dataset location. Replacing that secret changes the
dataset cache identity, so subsequent statements open with the new
credentials.
Output:
┌───────┬─────────┬─────────────────────┐
│ id │ name │ email │
│ int32 │ varchar │ varchar │
├───────┼─────────┼─────────────────────┤
│ 1 │ Alice │ alice@example.com │
│ 2 │ Bob │ bob@example.com │
│ 3 │ Charlie │ charlie@example.com │
└───────┴─────────┴─────────────────────┘
Output:
┌───────┬─────────┬─────────────────────┐
│ id │ name │ email │
│ int32 │ varchar │ varchar │
├───────┼─────────┼─────────────────────┤
│ 2 │ Bob │ bob@example.com │
│ 3 │ Charlie │ charlie@example.com │
└───────┴─────────┴─────────────────────┘
Output:
6. Using Fully Qualified Names¶
You can also use fully qualified table names without switching databases: