LanceDB Feature Catalog

This list outlines all the features offered by LanceDB - across OSS, Cloud and Enterprise product lines. Click on each feature to be taken to its individual documentation page.

Storage

LanceDB provides flexible storage backends that support both cloud object storage and local high-performance storage for different deployment scenarios.

Feature Description OSS Cloud Enterprise
Object, File, Block Storage Support for AWS, GCS, Azure and S3-compatible vendors.
Local SSD/NVMe Storage Support for storage on customer’s custom servers.

Tables

LanceDB’s table abstraction provides ACID-compliant data management with schema evolution, versioning, and consistency guarantees for vector and scalar data.

Feature Description OSS Cloud Enterprise
Tables - CRUD Operations Basic API to create, read, update, drop tables.
Tables - Data Evolution Alter column schema, datatype, backfill + merge data
Tables - Versioning Append, overwrite, check versions + tag them.
Tables - Consistency Synchronize database with underlying storage.

Ingestion

LanceDB’s ingestion pipeline handles both vector embedding generation and data loading with support for multiple formats and efficient batch operations.

Feature Description OSS Cloud Enterprise
Embedding - Text Data Generate vector embeddings from text data using various embedding models.
Embedding - Multimodal Data Generate embeddings from images, audio, and other multimodal content.
Embedding - CPU & GPU Device Configuration Configure CPU or GPU acceleration for embedding generation performance.
Embedding - Environment Variables Manage API keys and configuration for embedding model access.
Data Ingestion - Default Formerly called Adding Data to a Table.
Data Ingestion - Formats Pandas, Polars, Pyarrow, Pydantic
Data Ingestion - Upsert Update existing records or insert new ones based on key.
Data Ingestion - Merge Insert Combine data from multiple sources into a single table.

Indexing

LanceDB’s indexing system provides multiple vector and scalar index types with automated optimization for fast similarity search and retrieval operations.

Feature Description OSS Cloud Enterprise
Vector Index - IVF_FLAT Minimal index that looks at IVF partitions, instead of brute forcing.
Vector Index - IVF_PQ Default vector index using Euclidean distance.
Vector Index - IVF_SQ IVF index built using scalar quantized vectors.
Vector Index - IVF_HNSW_SQ HNSW built on IVF’s partitions + vectors that are scalar quantized.
Vector Index - Binary IVF_FLAT with Hamming distance for binary vectors.
Scalar Index BTREE, BITMAP, LABEL_LIST
Automated Indexing Indexing happens in the background no config.
Bypass Automated Indexing When you want to search over all available vectors.
Reindexing - Manual User needs to specify that they want to reindex.
Reindexing - Automated Reindexing happens in the background no config
GPU Indexing - Manual User needs to specify which indexing device to use.
GPU Indexing - Automated Indexing device is automatically set for user.
Full Text Search Index Inverted index

LanceDB’s search capabilities combine vector similarity search, full-text search, and hybrid approaches to provide comprehensive retrieval functionality across different data types.

Feature Description OSS Cloud Enterprise
Vector Search - No Index Goes through all the available vectors.
Vector Search - ANN Index Retrieves top K similar vectors.
Vector Search - Multivectors Late interaction vector search.
Vector Search - Distance Range Search for vectors within a specific distance threshold.
Vector Search - Binary Vectors Search using binary vector representations for efficiency.
Vector Search - Filtering Apply scalar filters during vector search operations.
Vector Search - Batch API Process multiple search queries in a single request.
Vector Search - Async Indexing Fallback brute force for fast performance.
Full Text Search - FTS Index Inverted Index
Full Text Search - Tokenizer Ngram and other common methods of splitting text data.
Full Text Search - Scalar Index BTREE, BITMAP, LABEL_LIST for non-vector data.
Full Text Search - Fuzzy Search Searching when there is a typo on the query.
Full Text Search - Prefix Matching Search for text that starts with specific characters.
Full Text Search - Score Boosting Increase relevance scores for specific terms or fields.
Full Text Search - Boolean Logic Use AND, OR, NOT operators in text search queries.
Full Text Search - Array Fields Search within array or list data types.
Hybrid Search - FTS Index Combine vector and full-text search in single query.
Hybrid Search - Reranking Reorder search results using additional ranking models.
SQL Queries Execute standard SQL queries on LanceDB tables.
Query Optimization Explain query plan, analyze query plan, optimization config settings.

Filtering

LanceDB’s filtering system provides flexible query capabilities that can be applied independently or in combination with vector and full-text search operations.

Feature Description OSS Cloud Enterprise
Filtering - no Vector Search Apply filters without vector search operations.
Filtering - Vector Search Apply filters during vector search operations.
Filtering - Full Text Search Apply filters during full-text search operations.