Why Multimodal Data Needs a Better Lakehouse? — Download the Research Study
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
Searching in Blog Posts
Feature Engineering for Multimodal Data: From Laptop to Cluster with LanceDB
This is some text inside of a div block.
Run a multimodal feature-engineering pipeline in ten minutes with geneva-examples, then rebuild its CLIP embedding step with LanceDB's feature engineering API, the same calls you'd use to enrich your own tables on LanceDB Enterprise.
⚡ Multi-Bit RaBitQ Without Refine, 🌋 Bytedance’s Lance-Based AI Stack, 🤖 Lance for Embodied AI Data
This is some text inside of a div block.
Multi-bit RaBitQ hits 96% recall without refine overhead, Volcano Engine cuts training pipelines from 7 days to 1, Lance powers 100 Hz robotics data ingestion, plus upcoming events, enterprise updates, community updates.
Why CrewAI Rebuilt Agent Memory on LanceDB, Powering 2B+ Agent Executions
This is some text inside of a div block.
CrewAI replaced a two-system memory stack with LanceDB, then rebuilt agent memory to resolve contradictions and gate recall on confidence. It now ships by default across 12 million monthly downloads and 2B+ agent executions.
How ByteDance’s Volcano Engine Rebuilt Its AI Stack on Lance, From Data Lake to Agent Memory at 100K+ QPS
This is some text inside of a div block.
Volcano Engine rebuilt its AI data stack on Lance, cutting a seven-day pipeline to one day and powering agent memory with LanceDB at 100K+ QPS.
Make Handwritten Notes Searchable: Optimizing an OCR Pipeline with LanceDB
This is some text inside of a div block.
Learn how to build an OCR pipeline for handwritten medical notes using DSPy, GEPA, and LanceDB to manage images, labels, outputs, metrics, and retrieval.
Rebuilding the Data Foundation for Embodied AI with Lance: From Long Videos to Random-Access-Friendly Multimodal Samples
This is some text inside of a div block.
Lance turns long robotics videos into random-access multimodal training data with 1.7–6× faster reads and 42% lower storage use.
RaBitQ Gets Faster: Higher Recall, Lower Latency, Query-Time Control
This is some text inside of a div block.
How LanceDB's latest IVF_RQ improvements raise recall, cut p99 latency, and let teams tune vector search at query time with approx_mode.
📊 Lance vs Delta vs Iceberg, 🔗 Lance Blob V2 Late Materialization, 🤖 Stable-Worldmodel Research Platform
This is some text inside of a div block.
Benchmark Lance vs Delta vs Iceberg on S3 metadata performance, update blob rows without reading bytes, and train world models directly from object storage, plus upcoming events and enterprise and community updates.
From Messy PDFs to Verifiable Answers with LiteParse and LanceDB
This is some text inside of a div block.
Turn information-rich PDFs into a local, inspectable, searchable evidence store with LlamaIndex's LiteParse library and LanceDB.
Faster VLM Fine-Tuning With Materialized Model Features in LanceDB
This is some text inside of a div block.
How Lance format and LanceDB's Enterprise feature engineering platform make VLM fine-tuning faster by materializing expensive multimodal features once and training from those columns.
Lance Blob V2: Late Materialization for Large Binary Data in Spark
This is some text inside of a div block.
How late materialization in Lance Spark keeps large binary data as lightweight references through query plans, materializing bytes only on write.
Semantic Memory for Hermes Agent with LanceDB
This is some text inside of a div block.
Introducing a new LanceDB-backed memory plugin that gives Hermes Agent durable, semantic recall across sessions, with benchmarks and a hands-on remember/recall/forget walkthrough.
A Metadata Benchmark of Lance, Delta Lake, and Iceberg on S3
This is some text inside of a div block.
A Rust benchmark comparison of Lance, Delta Lake, and Apache Iceberg on S3 and S3 Express, and why Lance is optimized for object storage metadata.
Scalable Feature Engineering on Multimodal Datasets
This is some text inside of a div block.
How LanceDB uses the Lance format's flexible data evolution features to enable scalable feature engineering for multimodal datasets.
Stable-Worldmodel: A High Performance Platform for Reproducible World Model Research
This is some text inside of a div block.
Introducing stable-worldmodel, an open-source platform for reproducible world model research, evaluation, and benchmarking under visual and physical distribution shifts.
Reproducible Data Curation In The Multimodal Lakehouse
This is some text inside of a div block.
Learn how LanceDB turns raw multimodal data into reproducible, training-ready datasets with search, filtering, deduplication, sampling, and versioned curation workflows.
🌍 Lance-Backed World Model Platform, 🦆 Multimodal SQL with Lance DuckDB Extension, 💰 LanceDB vs OpenSearch Cost Breakdown
This is some text inside of a div block.
stable-worldmodel standardizes world model pipelines on Lance, DuckDB Lance extension adds native multimodal SQL, and LanceDB benchmarks 100M vectors at ~$779/month, plus upcoming events, enterprise updates, and community updates.
⚡Vector Search at 10B Scale, 📊 Lance Format Benchmarks, 🚗 AV Pipelines at Scale
This is some text inside of a div block.
Distributed vector search at 10B scale, more efficient storage with Lance format v2.2, and production AV pipelines simplified, plus upcoming events and community updates.
How LanceDB Accelerates Vector Search at 10 Billion Scale
This is some text inside of a div block.
How LanceDB applies distributed indexing, distributed query execution, HNSW centroid routing, and fast RaBitQ rotation to scale search to 10B vectors and beyond.
OpenSearch vs LanceDB for Vector Search: Query Cost and Infrastructure
This is some text inside of a div block.
Choosing a vector database usually comes down to a tradeoff between a full search service and an in-process library. This post showcases benchmarks that compare OpenSearch and LanceDB on the COCO 2017 images embedded with SigLIP. We measure ingestion throughput, query cost, storage layout, and overall infra cost.
Volcano Engine LAS's Lance-Based PB-Scale Autonomous Driving Data Lake Solution
This is some text inside of a div block.
How Bytedance Volcano Engine LAS (Lake for AI Service) leverages Lance as the core storage format, rapidly constructing a next-gen AI data lake to efficiently store, manage, and process multimodal data (text, images, audio/video).
Unifying the AV ML Stack: From Raw Data to Trained Model with LanceDB
This is some text inside of a div block.
A complete walkthrough of building an autonomous vehicle perception model training pipeline on top of LanceDB and the Multimodal Lakehouse.
Lance JSON Support: Why You Might Not Really Need Variant
This is some text inside of a div block.
Lance's JSONB storage, scalar indexing, data evolution, and full-text search already deliver what most users want from Variant — with explicit control, schema consistency, and no vendor lock-in.
Building A Storage Format For The Next Era of Biology
This is some text inside of a div block.
How Lance can serve as the foundation for AI on single-cell genomics atlases and a new generation for modeling in biology.
📄 Lance Blob V2, 🤗 Upload Lance Datasets to HF Hub, 🦞 LanceDB for OpenClaw's Memory
This is some text inside of a div block.
Lance Blob V2 introduces adaptive storage semantics, easily upload Lance datasets to Hugging Face Hub, and OpenClaw establishes LanceDB as a default memory layer for agents, plus community and enterprise updates.
Smart Parsing Meets Sharp Retrieval: Combining LiteParse and LanceDB
This is some text inside of a div block.
Build a structure-aware PDF QA agent with LiteParse, LanceDB, and Claude to answer complex questions over visually rich documents.
Lance Format v2.2 Benchmarks: Half the Storage, None of the Slowdown
This is some text inside of a div block.
Benchmarks that show how Lance format v2.2 cuts storage by 50%+, beats Parquet on compression, and delivers up to 68x faster blob reads — while preserving the scan and random access patterns that multimodal training depends on.
Make your SQL Workflows Multimodal With LanceDB × DuckDB
This is some text inside of a div block.
A hands-on walkthrough showing how LanceDB and DuckDB work together to query multimodal data in SQL, join against multiple tables, and materialize results back into LanceDB.
Agentic Coding as Community Stewardship
This is some text inside of a div block.
Agentic coding lets upstream maintainers turn ecosystem-wide mechanical updates into fast, contextual, low-friction contributions for downstream users.
One System, Many Workloads: Rethinking What "Multimodal" Means for AI
This is some text inside of a div block.
A practical definition of multimodal complexity, and how LanceDB’s Multimodal Lakehouse is built to address these challenges.
The Future of AI-Native Development is Local: Inside Continue's LanceDB-Powered Evolution
This is some text inside of a div block.
Discover how Continue revolutionized AI-native development with LanceDB's embedded TypeScript library, enabling lightning-fast semantic code search while maintaining complete developer privacy and offline capability.
Lance File Format 2.2: Taming Complex Data
This is some text inside of a div block.
Lance file format 2.2 introduces Blob V2, nested schema evolution, native Map type support, and additional compression and performance improvements for AI/ML data workloads.
Lance Blob V2: Making Multimodal Data a First-Class Citizen in the Lakehouse
This is some text inside of a div block.
How we redesigned blob storage in Lance to make multimodal data a first-class citizen, with four storage semantics (Inline, Packed, Dedicated, External) that automatically adapt to your workload.
Why LanceDB Is the Most Natural Memory Layer for OpenClaw
This is some text inside of a div block.
OpenClaw and similar personal autonomous agents need a local-first long-term memory layer. LanceDB fits that role with embedded deployment, filesystem-native storage, and multimodal retrieval.
OpenClaw + LanceDB + Seed 2.0: Turn Visual Ideas into Reality, Fast!
This is some text inside of a div block.
OpenClaw and similar personal autonomous agents need a local-first long-term memory layer. LanceDB fits that role with embedded deployment, filesystem-native storage, and multimodal retrieval.
Memory for OpenClaw: From Zero to LanceDB Pro
This is some text inside of a div block.
Benchmarking three OpenClaw memory plugins on the LOCOMO dataset
A Guide to Uploading Lance Datasets on the Hugging Face Hub
This is some text inside of a div block.
Build a multimodal Lance dataset, publish it to the Hub, and query the precomputed vector + FTS indexes in LanceDB, without needing to download the dataset locally.
Zero Shot Image Classification with Vector Search
This is some text inside of a div block.
Get about zero shot image classification with vector search. Get practical steps, examples, and best practices you can use now.
WeRide's Data Platform Transformation: How LanceDB Fuels Model Development Velocity
This is some text inside of a div block.
Discover how WeRide, a leading autonomous driving company, leveraged LanceDB to revolutionize their data platform, achieving 90x improvement in ML developer productivity and reducing data mining time from 1 week to 1 hour.
Training a Variational AutoEncoder from Scratch with Lance File Format
This is some text inside of a div block.
Train a Variational Autoencoder end‑to‑end using Lance for fast, scalable data handling. You’ll set up the dataset, build the VAE in PyTorch, and run training, sampling, and reconstructions.
Track AI Trends: CrewAI Agents & RAG
This is some text inside of a div block.
This article will teach us how to make an AI Trends Searcher using CrewAI Agents and their Tasks. But before diving into that, let's first understand what CrewAI is and how we can use it for these applications.
Tokens per Second Is NOT All You Need
This is some text inside of a div block.
Explore about tokens per second is not all you need. Get practical steps, examples, and best practices you can use now.
The Future of Open Source Table Formats: Apache Iceberg and Lance
This is some text inside of a div block.
Explore the future of open source table formats: apache iceberg and lance with practical insights and expert guidance from the LanceDB team.
The Case for Random Access I/O
This is some text inside of a div block.
One of the reasons we started the Lance file format and have been investigating new encodings is because we wanted a format with better support for random access.
LanceDB Raises $30M Series A to Build the Multimodal Lakehouse
This is some text inside of a div block.
We have closed another funding round to accelerate development of the Multimodal Lakehouse - a unified platform for AI data infrastructure.
SemanticDotArt: Rethinking Art Discovery with LanceDB
This is some text inside of a div block.
SemanticDotArt turns art discovery into a multimodal search experience, matching feelings, phrases, and images with LanceDB's hybrid retrieval.
Second Dinner's Secret Weapon: LanceDB-Powered RAG for Faster, Smarter Game Development
This is some text inside of a div block.
Discover how Second Dinner, creators of Marvel Snap, leveraged LanceDB Cloud to transform game development workflows, reducing prototyping time from months to hours and automating QA test generation with 81% better results.
Search Within an Image with Segment Anything
This is some text inside of a div block.
Get about search within an image with segment anything. Get practical steps, examples, and best practices you can use now.
Scalable Computer Vision with LanceDB & Voxel51
This is some text inside of a div block.
Explore about scalable computer vision with lancedb & voxel51. Get practical steps, examples, and best practices you can use now.
Rethinking Table File Paths with Uber: Lance’s Multi-Base Layout
This is some text inside of a div block.
A tour of Lance's file path design, and how Lance’s new multi-base layout enables multi-location datasets (such as Uber’s multi-bucket setup) with minimal metadata rewrites.
RAG Isn't One-Size-Fits-All: Here's How to Tune It for Your Use Case
This is some text inside of a div block.
Great RAG comes from a tight iteration loop. Learn how to systematically improve each layer of your RAG system using Kiln and LanceDB.
Python Package to convert image datasets to Lance type
This is some text inside of a div block.
Explore python package to convert image datasets to lance type with practical insights and expert guidance from the LanceDB team.
The Quest for One Million IOPS: Benchmarking Storage at LanceDB
This is some text inside of a div block.
Learn how LanceDB benchmarks storage and how we achieved one million disk reads per second.
November Feature Roundup
This is some text inside of a div block.
Explore November feature roundup with practical insights and expert guidance from the LanceDB team.
🛡️ Newly Knighted Lancelot, ▶️ TwelveLabs Semantic Video Recommendations, 🧠 Cognee's AI Memory Layer with LanceDB
This is some text inside of a div block.
Our September newsletter welcomes new Lancelot members, highlights TwelveLabs semantic video recommendations, Cognee’s AI memory layer, and shares the latest product and community updates.
🎨 Semantic.Art, 💾 Stable Lance 2.1, 🎥 Ray+LanceDB powers Netflix
This is some text inside of a div block.
Our October newsletter highlights Semantic.Art, Lance File 2.1, RaBitQ Quantization, upcoming events, latest product and community updates.
🛡️Lance Community Governance, Lance + Iceberg 🧊, Netflix’s Multimodal Search Demo 🔍
This is some text inside of a div block.
Our November newsletter highlights Lance community governance, a deep dive on Lance and Iceberg, a demo of Netflix's multimodal search, previous talk recordings, and the latest product and community updates.
June 2025: $30M Series A, Multimodal Lakehouse Launch & Product Updates
This is some text inside of a div block.
LanceDB's June 2025 newsletter covering latest company news, product updates, open source releases, and community highlights.
⚖️ Harvey’s Enterprise-Grade RAG on LanceDB, 💼 Dosu Case Study, Minimax&LumaLabs❤️Lance-Ray
This is some text inside of a div block.
Our August newsletter features a new case study with Dosu, recaps from events with Harvey and Databricks, and the latest product and community updates.
🦆 Lance x DuckDB SQL Retrieval, 🚗 Uber-Scale Storage, ⚡ 1.5M IOPS
This is some text inside of a div block.
Kicking off 2026 with Lance-native SQL retrieval via DuckDB, Uber-scale multi-bucket storage, 1.5M IOPS benchmarks, and continued OSS momentum across the Lance ecosystem.
🤗 Lance x Hugging Face, 🪾Git-Style Branching, 🏔️ Geospatial in Lance
This is some text inside of a div block.
Native Lance support on Hugging Face Hub, Git-style branching and shallow clone for AI data, and Arrow-native geospatial with R-Tree indexing, plus steady OSS and community momentum.
💾 Lance SDK v1.0.0, 🗓️ 1st Lance Community Sync, 🔍 Wikisearch
This is some text inside of a div block.
Our December newsletter highlights Lance SDK v1.0.0, our upcoming Lance community sync, Wikisearch demo, and the latest product and community updates.
Netflix’s Media Data Lake ❤️ LanceDB, CodeRabbit 💼 Case Study, Lance Namespace
This is some text inside of a div block.
Our August newsletter highlights LanceDB powering Netflix's Media Data Lake, a case study on CodeRabbit's AI-powered code reviews, and updates on Lance Namespace and Spark integration.
My Summer Internship Experience at LanceDB
This is some text inside of a div block.
I'm Raunak, a master's student at the University of Illinois, Urbana-Champaign. This summer, I had the opportunity to intern as a Software Engineer at LanceDB, an early-stage startup based in San Francisco.
My SIMD Is Faster than Yours
This is some text inside of a div block.
An untold story about how we make LanceDB vector search fast. Get practical steps and examples from 'My SIMD is faster than Yours'.
Multimodal Myntra Fashion Search Engine Using LanceDB
This is some text inside of a div block.
Build a multimodal fashion search engine with LanceDB and CLIP embeddings. Follow a step‑by‑step workflow to register embeddings, create the table, query by text or image, and ship a Streamlit UI.
What is the LanceDB Multimodal Lakehouse?
This is some text inside of a div block.
Introducing the Multimodal Lakehouse - a unified platform for managing AI data from raw files to production-ready features, now part of LanceDB Enterprise.
Multi Document Agentic RAG: a Walkthrough
This is some text inside of a div block.
Unlock about multi document agentic rag: a walkthrough. Get practical steps, examples, and best practices you can use now.
Modified RAG: Parent Document & Bigger Chunk Retriever
This is some text inside of a div block.
Get about modified rag: parent document & bigger chunk retriever. Get practical steps, examples, and best practices you can use now.
MemGPT: OS Inspired LLMs That Manage Their Own Memory
This is some text inside of a div block.
Explore about memgpt: os inspired llms that manage their own memory. Get practical steps, examples, and best practices you can use now.
Late Interaction & Efficient Multi-modal Retrievers Need More Than a Vector Index
This is some text inside of a div block.
Explore late interaction & efficient multi-modal retrievers need more than a vector index with practical insights and expert guidance from the LanceDB team.
Developers, Ditch the Black Box: Welcome to Continue
This is some text inside of a div block.
Remember flipping through coding manuals? Those quickly became relics with the rise of Google and Stack Overflow, a one-stop shop for developer queries.
Lance × Hugging Face: A New Era of Sharing Multimodal Data on the Hub
This is some text inside of a div block.
Announcing native read support for Lance format on Hugging Face Hub. You can now distribute your large multimodal datasets as a single, searchable artifact (including blobs, embeddings and indexes) all in one place!
Lance × DuckDB: SQL for Retrieval on the Multimodal Lakehouse Format
This is some text inside of a div block.
Use the Lance format as your lakehouse layer for retrieval, RAG and more, with the native Lance extension for DuckDB
Lance, Windows. Windows, Lance
This is some text inside of a div block.
It was Spring of 2012. After being an avid user for 2+ years, I finally decided to join Wes Mckinney and work on pandas full time.
Lance v2: A New Columnar Container Format
This is some text inside of a div block.
Explore lance v2: a new columnar container format with practical insights and expert guidance from the LanceDB team.
Productionalize AI Workloads with Lance Namespace, LanceDB, and Ray
This is some text inside of a div block.
Learn how to productionalize AI workloads with Lance Namespace's enterprise stack integration and the scalability of LanceDB and Ray for end-to-end ML pipelines.
Lance File 2.1 is Now Stable
This is some text inside of a div block.
The 2.1 file version is now stable, learn what that means for you and what's coming next.
Lance File 2.1: Smaller and Simpler
This is some text inside of a div block.
Explore lance file 2.1: smaller and simpler with practical insights and expert guidance from the LanceDB team.
Introducing Lance Data Viewer: A Simple Way to Explore Lance Tables
This is some text inside of a div block.
A lightweight open source web UI for exploring Lance datasets, viewing schemas, and browsing table data with vector visualization support.
Building the Future Together: Introducing Lance Community Governance
This is some text inside of a div block.
Announcing the formal governance structure for the Lance community, establishing clear pathways for contribution and leadership with a three-tier system of contributors, maintainers and PMC.
Manage Lance Tables in Any Catalog using Lance Namespace and Spark
This is some text inside of a div block.
Access and manage your Lance tables in Hive, Glue, Unity Catalog, or any catalog service using Lance Namespace with the latest Lance Spark connector.
Implementing Corrective RAG in the Easiest Way
This is some text inside of a div block.
Even though text-generation models are good at generating content, they sometimes need to improve in returning facts. This happens because of the way they are trained.
Hybrid Search: RAG for Real-Life Production-Grade Applications
This is some text inside of a div block.
Get about hybrid search: rag for real-life production-grade applications. Get practical steps, examples, and best practices you can use now.
Hybrid Search: Combining BM25 and Semantic Search for Better Results with Langchain
This is some text inside of a div block.
Have you ever thought about how search engines find exactly what you're looking for? They usually use a mix of matching specific words and understanding the meaning behind them.
Hybrid Search and Custom Reranking with LanceDB
This is some text inside of a div block.
Combine keyword and vector search for higher‑quality results with LanceDB. This post shows how to run hybrid search and compare rerankers (linear combination, Cohere, ColBERT) with code and benchmarks.
Reduce Hallucinations from LLM-Powered Agents Using Long-Term Memory
This is some text inside of a div block.
Understand about reduce hallucinations from llm-powered agents using long-term memory. Get practical steps, examples, and best practices you can use now.
Implement Contextual Retrieval and Prompt Caching with LanceDB
This is some text inside of a div block.
Unlock about implement contextual retrieval and prompt caching with lancedb. Get practical steps, examples, and best practices you can use now.
RAG with GRPO Fine-Tuned Reasoning Model
This is some text inside of a div block.
Explore rag with grpo fine-tuned reasoning model with practical insights and expert guidance from the LanceDB team.
GraphRAG: Hierarchical Approach to Retrieval-Augmented Generation
This is some text inside of a div block.
Explore GraphRAG: hierarchical approach to retrieval-augmented-generation with practical insights and expert guidance from the LanceDB team.
GPU-Accelerated Indexing in LanceDB
This is some text inside of a div block.
Speed up vector index training in LanceDB with CUDA or Apple Silicon (MPS). See how GPU‑accelerated IVF/PQ training compares to CPU and how to enable it in code.
How We Added Geospatial Support To Lance With No New Code
This is some text inside of a div block.
How Lance's Arrow-native architecture enables first-class geospatial support through extension types, GeoDataFusion integration, and R-Tree indexing.
Building Semantic Video Recommendations with TwelveLabs and LanceDB
This is some text inside of a div block.
Build semantic video recommendations using TwelveLabs embeddings, LanceDB storage, and Geneva pipelines with Ray.
LanceDB's Geneva: Scalable Feature Engineering
This is some text inside of a div block.
Learn how to build scalable feature engineering pipelines with Geneva and LanceDB. This demo transforms image data into rich features including captions, embeddings, and metadata using distributed Ray clusters.
From BI to AI: A Modern Lakehouse Stack with Lance and Iceberg
This is some text inside of a div block.
A comparison of where Iceberg and Lance sit in the modern lakehouse stack. We highlight emerging architectures that are bridging the worlds of analytics and AI/ML workloads using these two formats, while being built on the same data foundation.
Setup Real-Time Multimodal AI Analytics with Apache Fluss (incubating) and Lance
This is some text inside of a div block.
Learn how to build real-time multimodal AI analytics by integrating Apache Fluss streaming storage with Lance's AI-optimized lakehouse. This guide demonstrates streaming multimodal data processing for RAG systems and ML workflows.
Columnar File Readers in Depth: Parallelism without Row Groups
This is some text inside of a div block.
Explore columnar file readers in depth: column shredding with practical insights and expert guidance from the LanceDB team.
LanceDB's RaBitQ Quantization for Blazing Fast Vector Search
This is some text inside of a div block.
Introducing RaBitQ quantization in LanceDB for higher compression, faster indexing, and better recall on high‑dimensional embeddings.
LanceDB WikiSearch: Native Full-Text Search on 41M Wikipedia Docs
This is some text inside of a div block.
No more Tantivy! We stress-tested native full-text search in our latest massive-scale search demo. Let's break down how it works and what we did to scale it.
Efficient RAG with Compression and Filtering
This is some text inside of a div block.
Discover about efficient rag with compression and filtering. Get practical steps, examples, and best practices you can use now.
Effortlessly Loading and Processing Images with Lance: a Code Walkthrough
This is some text inside of a div block.
Working with large image datasets in machine learning can be challenging, often requiring significant computational resources and efficient data-handling techniques.
Designing a Table Format for ML Workloads
This is some text inside of a div block.
Explore designing a table format for ML workloads with practical insights and expert guidance from the LanceDB team.
Custom Datasets for Efficient LLM Training Using Lance
This is some text inside of a div block.
See about custom datasets for efficient llm training using lance. Get practical steps, examples, and best practices you can use now.
Creating a FinTech AI Agent From Scratch
This is some text inside of a div block.
Explore fintech ai agent from scratch with practical insights and expert guidance from the LanceDB team.
Convert Any Image Dataset to Lance
This is some text inside of a div block.
In our article, we explored the remarkable capabilities of the Lance format, a modern, columnar data storage solution designed to revolutionize the way we work with large image datasets in machine learning.
Columnar File Readers in Depth: Structural Encoding
This is some text inside of a div block.
Deep dive into LanceDB's dual structural encoding approach - mini-block for small data types and full-zip for large multimodal data. Learn how this optimizes compression and random access performance compared to Parquet.
Columnar File Readers in Depth: Repetition & Definition Levels
This is some text inside of a div block.
Explore columnar file readers in depth: repetition & definition levels with practical insights and expert guidance from the LanceDB team.
Columnar File Readers in Depth: Compression Transparency
This is some text inside of a div block.
Explore columnar file readers in depth: compression transparency with practical insights and expert guidance from the LanceDB team.
Columnar File Readers in Depth: Column Shredding
This is some text inside of a div block.
Explore columnar file readers in depth: column shredding with practical insights and expert guidance from the LanceDB team.
Columnar File Readers in Depth: Backpressure
This is some text inside of a div block.
Streaming data applications can be tricky. When you can read data faster than you can process the data then bad things tend to happen. The various solutions to this problem are largely classified as backpressure.
Columnar File Readers in Depth: APIs and Fusion
This is some text inside of a div block.
The API used to read files has evolved over time, from simple full table reads to batch reads and eventually to iterative record batch readers. Lance takes this a step further to return a stream of read tasks.
Chunking Techniques with Langchain and LlamaIndex
This is some text inside of a div block.
In our last blog, we talked about chunking and why it is necessary for processing data through LLMs. We covered some simple techniques to perform text chunking.
Chunking Analysis: Which is the right chunking approach for your language?
This is some text inside of a div block.
Explore chunking analysis: which is the right chunking approach for your language? with practical insights and expert guidance from the LanceDB team.
Chat with Your Stats Using Langchain Dataframe Agent & LanceDB Hybrid Search
This is some text inside of a div block.
In this blog, we’ll explore how to build a chat application that interacts with CSV and Excel files using LanceDB’s hybrid search capabilities.
Netflix's Media Data Lake and the Rise of the Multimodal Lakehouse
This is some text inside of a div block.
How Netflix built a Media Data Lake powered by LanceDB and the Multimodal Lakehouse to unify petabytes of media assets for machine learning pipelines.
Case Study: Meet Dosu - the Intelligent Knowledge Base for Software Teams and Agents
This is some text inside of a div block.
How Dosu uses LanceDB to transform codebases into living knowledge bases with real-time search and versioning.
How Cognee Builds AI Memory Layers with LanceDB
This is some text inside of a div block.
How Cognee uses LanceDB to deliver durable, isolated, and low-ops AI memory from local development to managed production.
Case Study: How CodeRabbit Leverages LanceDB for AI-Powered Code Reviews
This is some text inside of a div block.
How CodeRabbit leverages LanceDB-powered context engineering turns every review into a quality breakthrough.
Building RAG on codebases: Part 2
This is some text inside of a div block.
Building a Cursor-like @codebase RAG solution. Part 2 focuses on the generating embeddings and the retrieval strategy using a combination of techniques in LanceDB.
Building RAG on codebases: Part 1
This is some text inside of a div block.
Building a Cursor-like @codebase RAG solution. Part 1 focuses on indexing techniques, chunking strategies, and generating embeddings in LanceDB.
Branching and Shallow Cloning in Lance: Towards a "Git for AI Data"
This is some text inside of a div block.
A deep dive into how table formats handle version management for ML/AI experimentation, and how Lance unifies branching, tagging, and shallow clone on top of its multi-base architecture.
Better RAG with Active Retrieval Augmented Generation FLARE
This is some text inside of a div block.
by Akash A. Get practical steps and examples from 'Better RAG with Active Retrieval Augmented Generation FLARE'.
Benchmarking Random Access in Lance
This is some text inside of a div block.
In this short blog post we’ll take you through some simple benchmarks to show the random access performance of Lance format. Get practical steps and examples from 'Benchmarking random access in Lance'.
Inverted File Product Quantization (IVF_PQ): Accelerate Vector Search by Creating Indices
This is some text inside of a div block.
Compress vectors with PQ and accelerate retrieval with IVF_PQ in LanceDB. The tutorial explains the concepts, memory savings, and a minimal implementation with search tuning knobs.
Benchmarking Cohere Rerankers with LanceDB
This is some text inside of a div block.
Improve retrieval quality by reranking LanceDB results with Cohere and ColBERT. You’ll plug rerankers into vector, FTS, and hybrid search and compare accuracy on real datasets.
AnythingLLM's Competitive Edge: LanceDB for Seamless RAG and Agent Workflows
This is some text inside of a div block.
Discover how AnythingLLM leveraged LanceDB's serverless architecture to eliminate vector database setup complexity, enabling seamless cross-platform RAG and agent workflows with zero configuration required.
Announcing Lance SDK 1.0.0: What This Milestone Means for the Community
This is some text inside of a div block.
We’re excited to announce that the core Rust SDK and the Python and Java binding SDKs are graduating to version 1.0.0, alongside a new, community-driven release strategy.
Agentic RAG Using LangGraph: Build an Autonomous Customer Support Agent
This is some text inside of a div block.
Build an autonomous customer support agent using LangGraph and LanceDB that automatically fetches, classifies, drafts, and responds to emails with RAG-powered policy retrieval.
Advanced RAG: Precise Zero-Shot Dense Retrieval with HyDE
This is some text inside of a div block.
In the world of search engines, the quest to find the most relevant information is a constant challenge. Researchers are always on the lookout for innovative ways to improve the effectiveness of search results.
Accelerate Vector Search Applications Using OpenVINO & LanceDB
This is some text inside of a div block.
We show how to use the CLIP from OpenAI for Text-to-Image and Image-to-Image searching. We’ll also do a comparative analysis of the PyTorch model, FP16 OpenVINO format, and INT8 OpenVINO format in terms of speedup.
A Primer on Text Chunking and Its Types
This is some text inside of a div block.
Text chunking is a technique in natural language processing that divides text into smaller segments, usually based on the parts of speech and grammatical meanings of the words.
A Practical Guide to Training Custom Rerankers
This is some text inside of a div block.
Explore a practical guide to training custom rerankers with practical insights and expert guidance from the LanceDB team.
A Practical Guide to Fine-Tuning Embedding Models
This is some text inside of a div block.
Explore a practical guide to fine-tuning embedding models with practical insights and expert guidance from the LanceDB team.
Keep Your Data Fresh with CocoIndex and LanceDB
This is some text inside of a div block.
Store a multimodal dataset of recipes in LanceDB, a multimodal lakehouse for AI, and keep it fresh with CocoIndex, a declarative data transformation framework for AI with incremental processing capabilities.
No results found in blog posts
feature-engineering-examples
Justin Miller
newsletter-july-2026
ChanChan Mao
crewai-rebuilt-agent-memory-on-lancedb
CrewAI
one-table-to-train-your-robot-lancedb-as-the-data-layer-for-lerobot
Ayush Chaurasia
volcano-engine-lance-agent-memory
Bytedance
make-handwritten-notes-searchable-optimizing-an-ocr-pipeline-with-lancedb
Prashanth Rao
china-merchants-lancedb-story
China Merchants Lion Rock AI Lab
rabitq-gets-faster-higher-recall-lower-latency-query-time-control
Yang Cen
newsletter-june-2026
ChanChan Mao
from-messy-pdfs-to-verifiable-answers-with-liteparse-and-lancedb
Prashanth Rao
Clelia Astra Bertelli
faster-vlm-fine-tuning-with-materialized-model-features-in-lancedb
Prashanth Rao
Ayush Chaurasia
lance-blob-v2-late-materialization-for-large-binary-data-in-spark
Drew Gallardo
semantic-memory-for-hermes-agent-with-lancedb
Prashanth Rao
a-metadata-benchmark-of-lance-delta-lake-and-iceberg-on-s3
Jack Ye
scalable-feature-engineering-on-multimodal-datasets
Prashanth Rao
stable-worldmodel-a-high-performance-platform-for-reproducible-world-model-research
Ayush Chaurasia
Quentin Lhoest
Lucas Maes
Quentin Le Lidec
reproducible-data-curation-in-the-multimodal-lakehouse
Prashanth Rao
newsletter-may-2026
ChanChan Mao
newsletter-april-2026
ChanChan Mao
how-lancedb-accelerates-vector-search-at-10-billion-scale
Yang Cen
opensearch-vs-lancedb-for-vector-search-query-cost-and-infrastructure
Justin Miller
volcano-engine-autonomous-driving-data-lake-solution
Kejian Ju
unifying-the-av-ml-stack-lancedb
Ayush Chaurasia
lance-json-support-why-you-might-not-really-need-variant
Jack Ye
building-a-storage-format-for-the-next-era-of-biology
Pavan Ramkumar
newsletter-march-2026
ChanChan Mao
smart-parsing-meets-sharp-retrieval-combining-liteparse-and-lancedb
Clelia Astra Bertelli
Prashanth Rao
lance-format-v2-2-benchmarks-half-the-storage-none-of-the-slowdown
Xuanwo
make-your-sql-workflows-multimodal-with-lancedb-x-duckdb
Prashanth Rao
agentic-coding-as-community-stewardship
Xuanwo
what-we-mean-by-multimodal
Prashanth Rao
ai-native-development-local-continue-lancedb
Ty Dunn
lance-file-format-2-2-taming-complex-data
Xuanwo
lance-blob-v2
Xuanwo
Jack Ye
openclaw-lancedb-memory-layer
Xuanwo
Prashanth Rao
openclaw-lancedb-seed2
LanceDB
openclaw-memory-from-zero-to-lancedb-pro
Prashanth Rao
upload-lance-datasets-to-hf-hub
Prashanth Rao
zero-shot-image-classification-with-vector-search
Vipul Maheshwari
werides-data-platform-transformation-how-lancedb-fuels-model-development-velocity
Qian Zhu
Fei Chen
training-a-variational-autoencoder-from-scratch-with-the-lance-file-format
LanceDB
track-ai-trends-crewai-agents-rag
LanceDB
tokens-per-second-is-not-all-you-need
Mingran Wang
Tan Li
the-future-of-open-source-table-formats-iceberg-and-lance
Jack Ye
the-case-for-random-access-i-o
LanceDB
series-a-funding
Chang She
semanticdotart
Ayush Chaurasia
second-dinners-secret-weapon-lancedb-powered-rag-for-faster-smarter-game-development
Qian Zhu
search-within-an-image-331b54e4285e
Kaushal Choudhary
scalable-computer-vision-with-lancedb-voxel51-d8b65066d5f6
LanceDB
rethinking-table-file-paths-lance-multi-base-layout
Jack Ye
rag-isnt-one-size-fits-all
Leonard Marcq
python-package-to-convert-image-datasets-to-lance-type
Vipul Maheshwari
one-million-iops
Weston Pace
november-feature-roundup
Will Jones
newsletter-september-2025
Jasmine Wang
newsletter-october-2025
Jasmine Wang
newsletter-november-2025
ChanChan Mao
newsletter-june-2025
David Myriel
newsletter-july-2025
Jasmine Wang
newsletter-january-2026
ChanChan Mao
newsletter-february-2026
ChanChan Mao
newsletter-december-2025
ChanChan Mao
newsletter-august-2025
Jasmine Wang
my-summer-internship-experience-at-lancedb-2
Raunak Sinha
my-simd-is-faster-than-yours-fb2989bf25e7
LanceDB
multimodal-myntra-fashion-search-engine-using-lancedb
LanceDB
multimodal-lakehouse
David Myriel
multi-document-agentic-rag-a-walkthrough
Vipul Maheshwari
modified-rag-parent-document-bigger-chunk-retriever-62b3d1e79bc6
Mahesh Deshwal
memgpt-os-inspired-llms-that-manage-their-own-memory-793d6eed417e
Ayush Chaurasia
late-interaction-efficient-multi-modal-retrievers-need-more-than-just-a-vector-index
Ayush Chaurasia
lancedb-x-continue
LanceDB
lance-x-huggingface-a-new-era-of-sharing-multimodal-data
Prashanth Rao
Quentin Lhoest
Xuanwo
Ayush Chaurasia
lance-x-duckdb-sql-retrieval-on-the-multimodal-lakehouse-format
Xuanwo
lance-windows-windows-lance
Chang She
lance-v2
Weston Pace
lance-namespace-lancedb-and-ray
Jack Ye
lance-file-2-1-stable
Weston Pace
lance-file-2-1-smaller-and-simpler
Weston Pace
lance-data-viewer
Gordon Murray
lance-community-governance
Jack Ye
introducing-lance-namespace-spark-integration
Jack Ye
implementing-corrective-rag-in-the-easiest-way-2
LanceDB
hybrid-search-rag-for-real-life-production-grade-applications-e1e727b3965a
Mahesh Deshwal
hybrid-search-combining-bm25-and-semantic-search-for-better-results-with-lan-1358038fe7e6
LanceDB
hybrid-search-and-custom-reranking-with-lancedb-4c10a6a3447e
LanceDB
how-to-reduce-hallucinations-from-llm-powered-agents-using-long-term-memory-72f262c3cc1f
Tevin Wang
guide-to-use-contextual-retrieval-and-prompt-caching-with-lancedb
LanceDB
grpo-understanding-and-fine-tuning-the-next-gen-reasoning-model-2
Mahesh Deshwal
graphrag-hierarchical-approach-to-retrieval-augmented-generation
Akash Desai
gpu-accelerated-indexing-in-lancedb-27558fa7eee5
LanceDB
geo-support
Jack Ye
geneva-twelvelabs
David Myriel
geneva-feature-engineering
Jonathan Hsieh
from-bi-to-ai-lance-and-iceberg
Jack Ye
Prashanth Rao
fluss-integration
Wayne Wang
file-readers-in-depth-parallelism-without-row-groups
Weston Pace
feature-rabitq-quantization
David Myriel
Yang Cen
feature-full-text-search
David Myriel
enhance-rag-integrate-contextual-compression-and-filtering-for-precision-a29d4a810301
Kaushal Choudhary
effortlessly-loading-and-processing-images-with-lance-a-code-walkthrough
LanceDB
designing-a-table-format-for-ml-workloads
Weston Pace
custom-dataset-for-llm-training-using-lance
LanceDB
creating-a-fintech-agent
Vipul Maheshwari
convert-any-image-dataset-to-lance
LanceDB
columnar-file-readers-in-depth-structural-encoding
Weston Pace
columnar-file-readers-in-depth-repetition-definition-levels
Weston Pace
columnar-file-readers-in-depth-compression-transparency
Weston Pace
columnar-file-readers-in-depth-column-shredding
Weston Pace
columnar-file-readers-in-depth-backpressure
Weston Pace
columnar-file-readers-in-depth-apis-and-fusion
Weston Pace
chunking-techniques-with-langchain-and-llamaindex
Prashant Kumar
chunking-analysis-which-is-the-right-chunking-approach-for-your-language
Shresth Shukla
chat-with-csv-excel-using-lancedb
LanceDB
case-study-netflix
David Myriel
case-study-dosu
Qian Zhu
Michael Ludden
case-study-cognee
David Myriel
Vasilije Markovic
case-study-coderabbit
Qian Zhu
building-rag-on-codebases-part-2
Sankalp Shubham
building-rag-on-codebases-part-1
Sankalp Shubham
branching-and-shallow-clone
Jack Ye
better-rag-with-active-retrieval-augmented-generation-flare-3b66646e2a9f
LanceDB
benchmarking-random-access-in-lance
Chang She
benchmarking-lancedb-92b01032874a-2
LanceDB
benchmarking-cohere-reranker-with-lancedb
LanceDB
anythingllms-competitive-edge-lancedb-for-seamless-rag-and-agent-workflows
Ayush Chaurasia
announcing-lance-sdk
Weston Pace
agentic-rag-using-langgraph-building-a-simple-customer-support-autonomous-agent
LanceDB
advanced-rag-precise-zero-shot-dense-retrieval-with-hyde-0946c54dfdcb
LanceDB
accelerate-vector-search-applications-using-openvino-lancedb
LanceDB
a-primer-on-text-chunking-and-its-types-a420efc96a13
Prashant Kumar
a-practical-guide-to-training-custom-rerankers
Ayush Chaurasia
a-practical-guide-to-fine-tuning-embedding-models
Ayush Chaurasia
keep-your-data-fresh-with-cocoindex-and-lancedb
Prashanth Rao
Linghua Jin
Existing Privacy Settings Found
Submitted On
Mar 26, 2025 – 14:37:22 UTC
Your Consent ID
AMC89MQ329407NSADRFA09MA7SD
Required
Essential Consent
Off
On
Marketing Consent
Off
On
Analytics Consent
Off
On
Personalization Consent
LanceDB uses cookies and similar technologies to improve your experience, analyze traffic, and to show you relevant content and advertising.
  • You can accept all, reject all, or customize your privacy settings.
  • Non-essential cookies are disabled by default.
  • Closing this banner does not confirm any choice.
  • See our Privacy Policy for more information.
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
LanceDB uses cookies and similar technologies to improve your experience, analyze traffic, and to show you relevant content and advertising.

⚡ Multi-Bit RaBitQ Without Refine, 🌋 Bytedance’s Lance-Based AI Stack, 🤖 Lance for Embodied AI Data

ChanChan Mao
•
August 7, 2026
•
Newsletter
Table of Contents
This is a title
This is a subtitle

⚡ RaBitQ Gets Faster: Higher Recall, Lower Latency, Query-Time Control

High-recall vector search usually requires a refine step that re-ranks candidates against raw full-precision vectors — expensive in memory and tail latency. Multi-bit IVF_RQ in LanceDB now reaches 96.2% recall@10 with 5-bit codes at 2.6x lower p99 latency than unrefined IVF_PQ, no raw vectors needed.

New SIMD kernels and a fast rotation pass push throughput to 3.3x higher QPS per core. The approx_mode parameter lets you tune recall vs. latency at query time — fast, normal, or accurate — on the same index.

Read more →

💡 Case Study

How ByteDance’s Volcano Engine Rebuilt Its AI Stack on Lance, From Data Lake to Agent Memory at 100K+ QPS

Volcano Engine replaced single-node IVFPQ training (500GB+ memory, multi-day runs) with distributed index training across Lance fragments, cutting a 7-day model training pipeline to 1 day at 95% GPU utilization. Binary copy compaction skips decode/re-encode entirely, dropping compaction time from 418s to 15s on 5M-row tables.

ArkClaw, their managed OpenClaw deployment, runs memory-lancedb-ultra at 100K+ QPS with hybrid FTS+vector search and GitforMemory—agent memory branching built on Lance’s native branching API.

Volcano Engine’s Lance-Based AI Stack →

Rebuilding the Data Foundation for Embodied AI with Lance: From Long Videos to Random-Access-Friendly Multimodal Samples

China Merchants Lion Rock AI Lab rebuilt their robotics data pipeline on Lance to handle 100 Hz robot-arm states, multi-camera video streams, and per-frame annotations in a single table row. Long videos are sliced into GOP-sized blobs, preserving video-level compression while enabling frame-level random access without full-video decoding.

The result: 1.7–6.0× faster random reads than LeRobot, ~42% storage savings, and a single S3 copy that collection, processing, and training all read/write directly—eliminating the package→upload→download→reupload cycle.

Lance for Embodied AI Data →

📚 Also Published

📅 Upcoming Events

Actuate — Aug 18-19, 2026 · San Francisco, CA

LanceDB is sponsoring Actuate by Foxglove. Stop by Booth 15 to discuss data infrastructure for robotics—specifically how to make video, LiDAR, sensor data, and embeddings searchable for curation, training, and evaluation workflows.

We also have a lightning talk at 1pm on Aug 18 at the breakout stage!

Register →

Ray Summit — Aug 24-26, 2026 · San Francisco

LanceDB is sponsoring Ray Summit by Anyscale, stop by our booth!

🎤 How Exa Indexes the Entire Web: Ray Data and Lance at Billion-Document Scale

Lei Xu (LanceDB) · Hubert Yuan (Exa)

Catch our joint session with Exa covering how they index hundreds of billions of web pages using Lance and Ray Data—including dependency-graph-driven execution, fragment-level patching to avoid write amplification, and stateful model reuse via Ray Actors.

Register →

Composable Data Management Systems (CDMS) — September 4, 2026 · Boston, MA

🎤 The AI Frontier for Composable Data Systems

Weston Pace (LanceDB)

The talk explores how composable data systems with well-defined API boundaries and plugin architectures are positioned to support agentic workloads, covering architectural challenges like stability boundaries, business considerations around balancing enterprise and open source value, and the shift in development focus from implementation to review and design as AI-assisted contributions accelerate.

Register →

🏗️ LanceDB Enterprise Updates

Performance

  • Lock-free per-table freshness check — A lock-free atomic check replaces the per-table lock on read freshness, removing a scheduler handoff that capped throughput on busy tables; on a 126M-row table, ID-based row lookups at concurrency 64 rose from about 1,570 to 5,954 requests per second.
  • Full-text search index metadata caching — A new cache for full-text-search index snapshots avoids a storage read on every request; at 768 concurrent requests, a diagnostic benchmark measured 93.7% higher throughput and 56% lower median latency.
  • Faster result transfer for wide-column queries — Higher default connection-pool and buffer-pool sizes speed up result transfer between nodes; on wide-column take queries (150KB rows, 100 results, ~16MB per response), throughput rose 13.5% and median wire-transfer latency dropped from 226ms to 71ms.
  • Lower-contention index cache backend — A lower-contention backend for the index cache's memory tier removes per-read bookkeeping that scaled with core count; on a 320-core host under cache-heavy full-text-search load, throughput rose from 18.9 to 191.3 queries per second.
  • Faster distributed full-text-search planning — Distributed full-text-search query plans now decode index segment metadata synchronously instead of routing it through blocking-pool scheduling; in a controlled benchmark, throughput rose 27.9–31.1% and p99 latency fell 22.3–24.0% across query types.

Features

Feature Description
Job management Job views now support instant search across all fields, job cancellation, per-job failure reasons with retryability, archived job history, and a SQL SHOW JOBS statement, giving unified visibility into background jobs like index builds and cache prewarming alongside Feature Engineering jobs.
Table history Tables now have a history view showing every version with its schema changes and row count, with a diff against main for any version and pagination to scroll back through history.
Bring-your-own-bucket Enterprise clusters can bind to a customer-owned storage bucket or container instead of provisioned storage, covering the full path from cluster creation through the data plane, with single-writer ownership enforcement and an installer access preflight check.
Distributed ngram, bloom filter, r-tree, zone map & HNSW indexes The distributed indexer now supports building these five additional index types across multiple workers, broadening the set of index types that can be built at scale on large tables.

🌟 Open Source Releases

Project Description
Lance v8.0.0 – v9.0.0
Release notes		 • FTS v2 is now the default index format, with bulk MAXSCORE search for top-k disjunctions (#7512, #7603)
• Data overlay files enable in-place column updates without full fragment rewrites; compaction now triggers on overlay count (#7535, #7536, #7772)
• Streaming IVF k-means training for large datasets (#6913)
• Cold reads up to 8× faster via lazy column metadata loading (#7375)
• Object store metrics now published via the metrics crate and exposed through OpenTelemetry in Python (#7533, #7537)
• MemWAL adds row-level deletes via tombstones, prefiltered vector/FTS search, and snapshot-consistent fresh-tier membership (#7417, #7138, #7215)
LanceDB v0.34.0 – v0.36.0
Release notes		 • Branch diff and merge: promote columns added on a branch onto main via new client APIs (#3686)
• Elastic dataloader for PyTorch: iterable dataset that dynamically adjusts batch sizes based on memory pressure (#3509); remote tables now work with PyTorch dataloaders (#3432)
• OpenTelemetry metrics: Lance internal metrics now exposed via OTel in Python and Node bindings (#3609)
• OAuth authentication: new OAuth header provider for connecting to LanceDB Cloud with OAuth tokens (#3579, #3586)
lance-namespace-impls v0.4.1
Release notes		 • Lance tables in Hive Metastore are now stored as external tables, preventing accidental data deletion when dropping tables via HMS (#146)
lance-namespace v0.9.0
Release notes		 • Breaking: REST spec now includes response context, requiring updates to clients consuming the API (#358)
• Added tag field to DescribeTableRequest for table version/tag-based lookups (#345)
lance-context v0.5.1 – v0.6.5
Release notes		 • New RolloutDB for RL rollout storage: native RolloutStore with MemWAL ingest, server-id sharding, count/time-triggered WAL merges, and filtered/ordered trajectory reads (#124, #126, #142, #158)
• User-defined schemas via SchemaSpec and GenericStore, exposed across Rust core, HTTP server, client, and Python bindings (#218, #220)
• Control-plane master service with React UI for experiment browsing, centralized compaction/WAL-merge scheduler (etcd-backed HA), Prometheus /metrics endpoints, and stats-table observability (#137, #139, #140, #154)
lance-ray v0.5.0
Release notes		 • Distributed index building now supports IVF_RQ vector indexes (#5228), bitmap indexes (#5169), and ZoneMap indexes (#5214), with configurable num_segments for controlling parallelism (#5229)
• New add_columns_from enables distributed column addition across Ray workers (#4923); nested field paths now supported in Ray workflows (#5173)
• Vector search performance improved by sharing IVF centroids and PQ codebooks across Ray tasks (#4744) and reusing a global Ray pool with pickled dataset references (#5149, #5157)
lance-spark v0.6.0 – v0.7.1
Release notes		 • Spark 4.2 support added (#696)
• Full-text search SQL extension for querying Lance tables directly via Spark SQL (#501)
• DDL support for branch and tag operations, enabling version control workflows from Spark (#576, #654)
• Deferred index creation with WITH (train=false) for creating indexes without immediate training (#558)

🫶 Community Contributions

Thank you to contributors from Bytedance, Uber, Pinterest, Tencent, Baidu, Adobe, NVIDIA for improvements across storage, indexing, query execution, distributed processing, and ecosystem integrations in LanceDB, Lance, and the broader ecosystem.

Notable contributions this month:

  • @XuQianJin-Stars — Added GooseFS object store support and implemented namespace directory table operations (add/alter/drop columns, update, delete)
  • @zhangyue19921010 — Introduced RowAddrRemap structure to prevent OOM during compaction and migrated distributed BTree builds to segmented index framework
  • @ddupg — Added TOS (Volcengine) object store support and enabled distributed vector/FTS segment builds with direct commit paths
  • @tobocop2 — Enabled runtime SIMD dispatch for pre-Haswell x86_64 builds, expanding hardware compatibility for from-source installations
  • @valkum — Added Dict-to-value-type casting in alter_columns schema evolution and fixed datafusion filter coercion for dictionary-encoded columns
  • @wombatu-kun — Accelerated regex and infix LIKE queries using ngram indexes and exposed per-query I/O metrics on ANN operators
  • @Ali2Arslan — Enabled reading column min/max directly from ZoneMap without full scans and implemented single-flight scalar index opening
  • @beinan — Implemented FM-Index scalar index for exact substring search and added Java schema override for fragment writes
  • @gstamatakis95 — Added shared RaBitQ rotation for distributed IVF_RQ builds and eliminated HEAD calls when opening vector indexes
  • @LuciferYang — Added Spark 4.2 support and contributed zero-copy BFloat16Array construction for improved memory efficiency

A heartfelt thank you to our community contributors of Lance and LanceDB this past month:

@2dmurali • @a-agmon • @adibaadi • @aimanmalib • @alowator • @ar-maan05 • @bugwz • @chakshu-dhannawat • @charleshuang119 • @chuenchen309 • @ckarnell • @claydugo • @clearlove10-c • @coyasong • @cswpy • @danielmao1 • @dcfocus • @dentiny • @devteamaegis • @ebyhr • @ecthlion • @erandagan • @everysympathy • @expyron • @fangbo • @fanng1 • @farmerchillax • @fl0-m • @geserdugarov • @ghx5t-sol • @glitch-ux • @goutamadwant • @haochengliu • @haroldbenoit • @hashwnath • @hellower • @hfutatzhanghb • @huahuay • @hushengquan • @ivscheianu • @j7nhai • @jay-ju • @jiaoew1991 • @jiaqizho • @jo-migo • @joaquinhuigomez • @jsap0914 • @jtuglu1 • @julianyg • @kaan-simbe • @kobihikri • @lakshjain7 • @leepokai • @leohoare • @leoreeyang • @lixmgl • @malinjawi • @mansiverma897993 • @markmcd • @mateuszossgit • @mediamana • @missing-identity • @mmatczuk • @mocobeta • @moongtnt • @morales-t-netflix • @neo-x7 • @niraj-mx07 • @nuthalapativarun • @nyl3532016 • @omkar-334 • @plotor • @pranavachar01 • @prrao87 • @puchengy • @rtmalikian • @ryantqiu • @saitejabandaru-in • @sanskar-singh-2403 • @sapnilb15 • @say-5 • @sbrunk • @sezruby • @shizoqua • @skycutter • @skyshineb • @sohumt123 • @solaris-star • @spectual • @ssrheart425 • @stumpylog • @summaryzb • @sushanth012 • @timsaucer • @touch-of-grey • @u70b3 • @vibhujawa • @vitaliy-pikalo • @vortex-captain • @wayneadams • @wending-y • @whitewooood • @wirybeaver • @wulansari999 • @xingsuo-zbz • @xixigoodluck • @xloya • @xtangxtang • @xuxiaoqiang666 • @xuzha • @yanghua • @yangjunz • @yangshangqing95 • @yaodong-shen • @yesunbmh • @yeung108 • @yohahaha • @ytyky • @yuju-huang • @yuvalif • @ywu342 • @yyzhao2025 • @zhangyang0418 • @zlepper • @ztorchan

🤝 Lance Community Sync Recap

Community sync sessions this month covered several significant proposals and releases. On the proposal side, discussions included new low-level APIs, a blob session API, multi-table commit and branching scalability improvements, and a design for scaling to millions of fragments. The team announced the Lance 9.0.0 SDK release with a 10.0.0 beta incoming, highlighted a 7x FTS performance improvement, and introduced Lance Gatekeeper, an automated PR review bot. Additional topics included the lance.org website redesign with plans for community content, and a proposal to streamline the format-change voting process by moving votes into PRs with a shortened 72-hour window.

The next Lance Community Sync will take place on Thursday, August 13 @ 9am PT.

ChanChan Mao
Developer Relations @ LanceDB
GitHub
Linkedin

⚡ Multi-Bit RaBitQ Without Refine, 🌋 Bytedance’s Lance-Based AI Stack, 🤖 Lance for Embodied AI Data

ChanChan Mao
•
July 31, 2026
newsletter-july-2026

Feature Engineering for Multimodal Data: From Laptop to Cluster with LanceDB

Justin Miller
•
August 5, 2026
feature-engineering-examples

Why CrewAI Rebuilt Agent Memory on LanceDB, Powering 2B+ Agent Executions

CrewAI
•
July 23, 2026
crewai-rebuilt-agent-memory-on-lancedb