Example - Airbnb financial data search¶

The code below is an example of hybrid search, a search algorithm that combines FTS and vector search in LanceDB.

Let's get stared with an example. In this notebook we'll use Airbnb financial data documents to search for "the specific reasons for higher operating costs" in a particular year.

In [ ]:

# Setup
!pip install lancedb pandas langchain langchain_openai langchain-community pypdf openai cohere tiktoken sentence_transformers tantivy==0.20.1

# Setup !pip install lancedb pandas langchain langchain_openai langchain-community pypdf openai cohere tiktoken sentence_transformers tantivy==0.20.1

In [2]:

import os
import getpass

# Set your OpenAI API key
os.environ["OPENAI_API_KEY"] = getpass.getpass()

import os import getpass # Set your OpenAI API key os.environ["OPENAI_API_KEY"] = getpass.getpass()

··········

In [3]:

def pretty_print(docs):
    for doc in docs:
        print(doc + "\n\n")

def pretty_print(docs): for doc in docs: print(doc + "\n\n")

In [ ]:

from langchain_community.document_loaders import PyPDFLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter

# Load $ABNB's financial report. This may take 1-2 minutes since the PDF is large
sec_filing_pdf = "https://d18rn0p25nwr6d.cloudfront.net/CIK-0001559720/8a9ebed0-815a-469a-87eb-1767d21d8cec.pdf"

# Create your PDF loader
loader = PyPDFLoader(sec_filing_pdf)

# Load the PDF document
documents = loader.load()

# Chunk the financial report
text_splitter = RecursiveCharacterTextSplitter(chunk_size=1024, chunk_overlap=0)
docs = text_splitter.split_documents(documents)

from langchain_community.document_loaders import PyPDFLoader from langchain.text_splitter import RecursiveCharacterTextSplitter # Load $ABNB's financial report. This may take 1-2 minutes since the PDF is large sec_filing_pdf = "https://d18rn0p25nwr6d.cloudfront.net/CIK-0001559720/8a9ebed0-815a-469a-87eb-1767d21d8cec.pdf" # Create your PDF loader loader = PyPDFLoader(sec_filing_pdf) # Load the PDF document documents = loader.load() # Chunk the financial report text_splitter = RecursiveCharacterTextSplitter(chunk_size=1024, chunk_overlap=0) docs = text_splitter.split_documents(documents)

In [ ]:

from langchain_community.vectorstores import LanceDB
from langchain_openai import OpenAIEmbeddings
import lancedb


embedding_function = OpenAIEmbeddings()

db = lancedb.connect("~/langchain")

# Load the document into LanceDB
db = LanceDB.from_documents(docs, embedding_function, connection=db)

from langchain_community.vectorstores import LanceDB from langchain_openai import OpenAIEmbeddings import lancedb embedding_function = OpenAIEmbeddings() db = lancedb.connect("~/langchain") # Load the document into LanceDB db = LanceDB.from_documents(docs, embedding_function, connection=db)

In [ ]:

table = db._table
table.create_fts_index("text")

table = db._table table.create_fts_index("text")

In [19]:

table.to_pandas().head()

table.to_pandas().head()

Out[19]:

	vector	id	text	metadata
0	[-0.0016961554, -0.03531899, 0.011809787, -0.0...	5c66d086-0fed-4270-a91b-c2b67b3ed052	Table of Contents\nUNITED STATES\nSECURITIES A...	{'page': 0, 'source': 'https://d18rn0p25nwr6d....
1	[-0.021446472, -0.021045355, 0.010823516, -0.0...	ddcfa6b1-3de8-4933-a187-6aa7b7ae87b4	Class A common stock, par value $0.0001 per sh...	{'page': 0, 'source': 'https://d18rn0p25nwr6d....
2	[-0.020018686, -0.014233166, -0.010991167, -0....	c391b1e1-6f66-41f2-82ff-18db5a218303	this chapter) during the preceding 12 months (...	{'page': 0, 'source': 'https://d18rn0p25nwr6d....
3	[-0.019061018, -0.0022632438, -0.011158161, -0...	3e896a62-8631-4a54-86bd-ee2f69f3b373	Indicate by check mark whether the registrant ...	{'page': 0, 'source': 'https://d18rn0p25nwr6d....
4	[-0.015733723, -0.012287037, -0.004055117, -0....	47f5dd55-b3e7-4879-afba-5ca9eea7341b	As of June 30, 2022, the aggregate market valu...	{'page': 1, 'source': 'https://d18rn0p25nwr6d....

Vector Search¶

Average latency: 3.48 ms ± 71.6 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

In [30]:

str_query = "What are the specific factors contributing to Airbnb's increased operational expenses in the last fiscal year?"
query = embedding_function.embed_query(str_query)
docs = table.search(query, query_type="vector").limit(5).to_pandas()["text"].to_list()

str_query = "What are the specific factors contributing to Airbnb's increased operational expenses in the last fiscal year?" query = embedding_function.embed_query(str_query) docs = table.search(query, query_type="vector").limit(5).to_pandas()["text"].to_list()

In [31]:

pretty_print(docs)

pretty_print(docs)

In addition, the number of listings on Airbnb may decline as a result of a number of other factors affecting Hosts, including: the COVID-19 pandemic; enforcement or threatenedenforcement of laws and regulations, including short-term occupancy and tax laws; private groups, such as homeowners, landlords, and condominium and neighborhood
associations, adopting and enforcing contracts that prohibit or restrict home sharing; leases, mortgages, and other agreements, or regulations that purport to ban or otherwise restrict
home sharing; Hosts opting for long-term rentals on other third-party platforms as an alternative to listing on our platform; economic, social, and political factors; perceptions of trust
and safety on and off our platform; negative experiences with guests, including guests who damage Host property, throw unauthorized parties, or engage in violent and unlawful


Made Possible by Hosts, Strangers, AirCover, Categories, and OMG marketing campaigns and launches, a $67.9 million increase in our search engine marketing and advertising
spend, a $25.1 million increase in payroll-related expenses due to growth in headcount and increase in compensation costs, a $22.0 million increase in third-party service provider
expenses, and a $11.1 million increase in coupon expense in line with increase in revenue and launch of AirCover for guests, partially offset by a decrease of $22.9 million related to
the changes in the fair value of contingent consideration related to a 2019 acquisition.
General and Administrative
2021 2022 % Change
(in millions, except percentages)
General and administrative $ 836 $ 950 14 %
Percentage of revenue 14 % 11 %
General and administrative expense increased $114.0 million, or 14%, in 2022 compared to 2021, primarily due to an increase in other business and operational taxes of $41.3


Our success depends significantly on existing guests continuing to book and attracting new guests to book on our platform. Our ability to attract and retain guests could be materially
adversely affected by a number of factors discussed elsewhere in these “Risk Factors,” including:
• events beyond our control such as the ongoing COVID-19 pandemic, other pandemics and health concerns, restrictions on travel, immigration, trade disputes, economic
downturns, and the impact of climate change on travel including the availability of preferred destinations and the increase in the frequency and severity of weather-relatedevents, including fires, floods, droughts, extreme temperatures and ambient temperature increases, severe weather and other natural disasters, and the impact of other
climate change on seasonal destinations;
• political, social, or economic instability;


• Hosts failing to meet guests’ expectations, including increased expectations for cleanliness in light of the COVID-19 pandemic;• increased competition and use of our competitors’ platforms and services;
• Hosts failing to provide differentiated, high-quality, and an adequate supply of stays or experiences at competitive prices;
• guests not receiving timely and adequate community support from us;
• our failure to provide new or enhanced offerings, tiers, or features that guests value;
• declines or inefficiencies in our marketing efforts;• negative associations with, or reduced awareness of, our brand;
• actual or perceived discrimination by Hosts in deciding whether to accept a requested reservation;
• negative perceptions of the trust and safety on our platform; and
• macroeconomic and other conditions outside of our control affecting travel and hospitality industries generally.


Table of Contents
Airbnb, Inc.
Consolidated Statements of Operations
(in millions, except per share amounts)
Year Ended December 31,
2020 2021 2022
Revenue $ 3,378 $ 5,992 $ 8,399 
Costs and expenses:
Cost of revenue 876 1,156 1,499 
Operations and support 878 847 1,041 
Product development 2,753 1,425 1,502 
Sales and marketing 1,175 1,186 1,516 
General and administrative 1,135 836 950 
Restructuring charges 151 113 89 
Total costs and expenses 6,968 5,563 6,597 
Income (loss) from operations (3,590) 429 1,802 
Interest income 27 13 186 
Interest expense (172) (438) (24)
Other income (expense), net (947) (304) 25 
Income (loss) before income taxes (4,682) (300) 1,989 
Provision for (benefit from) income taxes (97) 52 96 
Net income (loss) $ (4,585)$ (352)$ 1,893 
Net income (loss) per share attributable to Class A and Class B common stockholders:
Basic $ (16.12)$ (0.57)$ 2.97 
Diluted $ (16.12)$ (0.57)$ 2.79

Hybrid Search¶

LanceDB support hybrid search with custom Rerankers. Here's the summary of latency numbers of some of the Reranking methods available

Let us now perform hybrid search by combining vector and FTS search results. First, we'll cover the default Reranker.

Linear Combination Reranker¶

LinearCombinationReranker(weight=0.7) is used as the default reranker for reranking the hybrid search results if the reranker isn't specified explicitly. The weight param controls the weightage provided to vector search score. The weight of 1-weight is applied to FTS scores when reranking.

Latency: 71 ms ± 25.4 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

In [32]:

docs = table.search(query_type="hybrid").vector(query).text(str_query).limit(5).to_pandas()["text"].to_list()

docs = table.search(query_type="hybrid").vector(query).text(str_query).limit(5).to_pandas()["text"].to_list()

In [33]:

pretty_print(docs)

pretty_print(docs)

In addition, the number of listings on Airbnb may decline as a result of a number of other factors affecting Hosts, including: the COVID-19 pandemic; enforcement or threatenedenforcement of laws and regulations, including short-term occupancy and tax laws; private groups, such as homeowners, landlords, and condominium and neighborhood
associations, adopting and enforcing contracts that prohibit or restrict home sharing; leases, mortgages, and other agreements, or regulations that purport to ban or otherwise restrict
home sharing; Hosts opting for long-term rentals on other third-party platforms as an alternative to listing on our platform; economic, social, and political factors; perceptions of trust
and safety on and off our platform; negative experiences with guests, including guests who damage Host property, throw unauthorized parties, or engage in violent and unlawful


“Initial Delivery Date”); provided that the Pricing Certificate for any fiscal year may be delivered on any date following the Initial DeliveryDate that is prior to the date that is 365 days following the last day of the preceding fiscal year, so long as such Pricing Certificate includes acertification that delivery of such Pricing Certificate on or before the Initial Delivery Date was not possible because (i) the informationrequired to calculate the KPI Metrics for such preceding fiscal year was not available at such time or (ii) the report of the KPI Metrics Auditor,if relevant, was not available at such time (the date of the Administrative Agent’s receipt thereof, each a “Pricing Certificate Date”). Upondelivery of a Pricing Certificate in respect of a fiscal year, (i) the Applicable Rate for the Loans incurred by the Borrower shall be increased ordecreased (or neither increased nor decreased), as applicable, pursuant to the Sustainability Margin Adjustment as set forth in the KPI MetricsCertificate


Made Possible by Hosts, Strangers, AirCover, Categories, and OMG marketing campaigns and launches, a $67.9 million increase in our search engine marketing and advertising
spend, a $25.1 million increase in payroll-related expenses due to growth in headcount and increase in compensation costs, a $22.0 million increase in third-party service provider
expenses, and a $11.1 million increase in coupon expense in line with increase in revenue and launch of AirCover for guests, partially offset by a decrease of $22.9 million related to
the changes in the fair value of contingent consideration related to a 2019 acquisition.
General and Administrative
2021 2022 % Change
(in millions, except percentages)
General and administrative $ 836 $ 950 14 %
Percentage of revenue 14 % 11 %
General and administrative expense increased $114.0 million, or 14%, in 2022 compared to 2021, primarily due to an increase in other business and operational taxes of $41.3


(c) If, for any fiscal year, either (i) no Pricing Certificate shall have been delivered for such fiscal year or (ii) the PricingCertificate delivered for such fiscal year shall fail to include the Diverse Supplier Spend Percentage or GHG Emissions Intensity for suchfiscal year, then the Sustainability Margin Adjustment will be positive 0.050% and/or the Sustainability Fee Adjustment will be positive0.010%, as applicable, in each case commencing on the last day such Pricing Certificate could have been delivered in accordance with theterms of clause (a) above (it being understood that, in the case of the foregoing clause (ii), the Sustainability Margin Adjustment or theSustainability Fee Adjustment will be determined in accordance with such Pricing Certificate to the extent the (A) Sustainability MarginAdjustment or the Sustainability Fee Adjustment is included in such Pricing Certificate and (B) the Administrative Agent has separatelyreceived the Diverse Supplier Spend Percentage and/or GHG Emissions


Our success depends significantly on existing guests continuing to book and attracting new guests to book on our platform. Our ability to attract and retain guests could be materially
adversely affected by a number of factors discussed elsewhere in these “Risk Factors,” including:
• events beyond our control such as the ongoing COVID-19 pandemic, other pandemics and health concerns, restrictions on travel, immigration, trade disputes, economic
downturns, and the impact of climate change on travel including the availability of preferred destinations and the increase in the frequency and severity of weather-relatedevents, including fires, floods, droughts, extreme temperatures and ambient temperature increases, severe weather and other natural disasters, and the impact of other
climate change on seasonal destinations;
• political, social, or economic instability;

Cohere Reranker¶

This uses Cohere's Reranking API to re-rank the results. It accepts the reranking model name as a parameter. By default it uses the english-v3 model but you can easily switch to a multi-lingual model.

Latency: 605 ms ± 78.1 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

In [34]:

# Free API key
os.environ["COHERE_API_KEY"] = getpass.getpass()

# Free API key os.environ["COHERE_API_KEY"] = getpass.getpass()

··········

In [35]:

from lancedb.rerankers import CohereReranker

reranker = CohereReranker()
docs = table.search(query_type="hybrid").vector(query).text(str_query).limit(5).rerank(reranker).to_pandas()["text"].to_list()

from lancedb.rerankers import CohereReranker reranker = CohereReranker() docs = table.search(query_type="hybrid").vector(query).text(str_query).limit(5).rerank(reranker).to_pandas()["text"].to_list()

In [36]:

pretty_print(docs)

pretty_print(docs)

Increased operating expenses, decreased revenue, negative publicity, negative reaction from our Hosts and guests and other stakeholders, or other adverse impacts from any of the
above factors or other risks related to our international operations could materially adversely affect our brand, reputation, business, results of operations, and financial condition.
In addition, we will continue to incur significant expenses to operate our outbound business in China, and we may never achieve profitability in that market. These factors, combined
with sentiment of the workforce in China, and China’s policy towards foreign direct investment may particularly impact our operations in China. In addition, we need to ensure that
our business practices in China are compliant with local laws and regulations, which may be interpreted and enforced in ways that are different from our interpretation, and/or create


Made Possible by Hosts, Strangers, AirCover, Categories, and OMG marketing campaigns and launches, a $67.9 million increase in our search engine marketing and advertising
spend, a $25.1 million increase in payroll-related expenses due to growth in headcount and increase in compensation costs, a $22.0 million increase in third-party service provider
expenses, and a $11.1 million increase in coupon expense in line with increase in revenue and launch of AirCover for guests, partially offset by a decrease of $22.9 million related to
the changes in the fair value of contingent consideration related to a 2019 acquisition.
General and Administrative
2021 2022 % Change
(in millions, except percentages)
General and administrative $ 836 $ 950 14 %
Percentage of revenue 14 % 11 %
General and administrative expense increased $114.0 million, or 14%, in 2022 compared to 2021, primarily due to an increase in other business and operational taxes of $41.3


• Hosts failing to meet guests’ expectations, including increased expectations for cleanliness in light of the COVID-19 pandemic;• increased competition and use of our competitors’ platforms and services;
• Hosts failing to provide differentiated, high-quality, and an adequate supply of stays or experiences at competitive prices;
• guests not receiving timely and adequate community support from us;
• our failure to provide new or enhanced offerings, tiers, or features that guests value;
• declines or inefficiencies in our marketing efforts;• negative associations with, or reduced awareness of, our brand;
• actual or perceived discrimination by Hosts in deciding whether to accept a requested reservation;
• negative perceptions of the trust and safety on our platform; and
• macroeconomic and other conditions outside of our control affecting travel and hospitality industries generally.


Table of Contents
Airbnb, Inc.
Consolidated Statements of Operations
(in millions, except per share amounts)
Year Ended December 31,
2020 2021 2022
Revenue $ 3,378 $ 5,992 $ 8,399 
Costs and expenses:
Cost of revenue 876 1,156 1,499 
Operations and support 878 847 1,041 
Product development 2,753 1,425 1,502 
Sales and marketing 1,175 1,186 1,516 
General and administrative 1,135 836 950 
Restructuring charges 151 113 89 
Total costs and expenses 6,968 5,563 6,597 
Income (loss) from operations (3,590) 429 1,802 
Interest income 27 13 186 
Interest expense (172) (438) (24)
Other income (expense), net (947) (304) 25 
Income (loss) before income taxes (4,682) (300) 1,989 
Provision for (benefit from) income taxes (97) 52 96 
Net income (loss) $ (4,585)$ (352)$ 1,893 
Net income (loss) per share attributable to Class A and Class B common stockholders:
Basic $ (16.12)$ (0.57)$ 2.97 
Diluted $ (16.12)$ (0.57)$ 2.79


Our success depends significantly on existing guests continuing to book and attracting new guests to book on our platform. Our ability to attract and retain guests could be materially
adversely affected by a number of factors discussed elsewhere in these “Risk Factors,” including:
• events beyond our control such as the ongoing COVID-19 pandemic, other pandemics and health concerns, restrictions on travel, immigration, trade disputes, economic
downturns, and the impact of climate change on travel including the availability of preferred destinations and the increase in the frequency and severity of weather-relatedevents, including fires, floods, droughts, extreme temperatures and ambient temperature increases, severe weather and other natural disasters, and the impact of other
climate change on seasonal destinations;
• political, social, or economic instability;

Relevance score is returned by Cohere API and is independent of individual FTS and vector search scores.

In [38]:

table.search(query_type="hybrid").vector(query).text(str_query).limit(5).rerank(reranker).to_pandas()

table.search(query_type="hybrid").vector(query).text(str_query).limit(5).rerank(reranker).to_pandas()

Out[38]:

	vector	id	text	metadata	_relevance_score
0	[0.0034929817, -0.024774546, 0.012623285, -0.0...	18d4a926-99d9-447f-8b57-264d7a148bd7	Increased operating expenses, decreased revenu...	{'page': 18, 'source': 'https://d18rn0p25nwr6d...	0.985328
1	[-0.0042489874, -0.005382498, 0.007190078, -0....	a91b3506-39a2-4b19-8409-08333d83a1c6	Made Possible by Hosts, Strangers, AirCover, C...	{'page': 62, 'source': 'https://d18rn0p25nwr6d...	0.979036
2	[0.0076079983, -0.013340506, 0.018701892, -0.0...	fcc532b9-347b-4e36-8ae8-5a2a726bf574	• Hosts failing to meet guests’ expectations, ...	{'page': 11, 'source': 'https://d18rn0p25nwr6d...	0.961606
3	[-0.008694107, -0.01993283, 0.014201017, -0.02...	72b844e2-cc93-4495-bb67-c2c1a1fd6532	Table of Contents\nAirbnb, Inc.\nConsolidated ...	{'page': 72, 'source': 'https://d18rn0p25nwr6d...	0.696578
4	[0.005813433, -0.028278675, 0.018041687, -0.02...	1694d5a5-7ece-40b8-8022-dc3fa9aaa05a	Our success depends significantly on existing ...	{'page': 11, 'source': 'https://d18rn0p25nwr6d...	0.500779

ColBERT Reranker¶

Colbert Reranker is powered by ColBERT model. It runs locally using the huggingface implementation.

Latency - 950 ms ± 5.78 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

Note: First query might be slow. It is recommended to reuse the Reranker objects as the models are cached. Subsequent runs will be faster on reusing the same reranker object

In [48]:

!pip install rerankers

!pip install rerankers

Collecting rerankers
  Downloading rerankers-0.6.0-py3-none-any.whl.metadata (28 kB)
Requirement already satisfied: pydantic in /usr/local/lib/python3.10/dist-packages (from rerankers) (2.9.2)
Requirement already satisfied: tqdm in /usr/local/lib/python3.10/dist-packages (from rerankers) (4.66.6)
Requirement already satisfied: annotated-types>=0.6.0 in /usr/local/lib/python3.10/dist-packages (from pydantic->rerankers) (0.7.0)
Requirement already satisfied: pydantic-core==2.23.4 in /usr/local/lib/python3.10/dist-packages (from pydantic->rerankers) (2.23.4)
Requirement already satisfied: typing-extensions>=4.6.1 in /usr/local/lib/python3.10/dist-packages (from pydantic->rerankers) (4.12.2)
Downloading rerankers-0.6.0-py3-none-any.whl (41 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/41.1 kB ? eta -:--:--
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 41.1/41.1 kB 2.7 MB/s eta 0:00:00
Installing collected packages: rerankers
Successfully installed rerankers-0.6.0

In [49]:

from lancedb.rerankers import ColbertReranker

reranker = ColbertReranker()
docs = table.search(query_type="hybrid").vector(query).text(str_query).limit(5).rerank(reranker).to_pandas()["text"].to_list()

from lancedb.rerankers import ColbertReranker reranker = ColbertReranker() docs = table.search(query_type="hybrid").vector(query).text(str_query).limit(5).rerank(reranker).to_pandas()["text"].to_list()

Loading ColBERTRanker model colbert-ir/colbertv2.0 (this message can be suppressed by setting verbose=0)
No device set
Using device cpu
No dtype set
Using dtype torch.float32
Loading model colbert-ir/colbertv2.0, this might take a while...

tokenizer_config.json:   0%|          | 0.00/405 [00:00<?, ?B/s]

vocab.txt:   0%|          | 0.00/232k [00:00<?, ?B/s]

tokenizer.json:   0%|          | 0.00/466k [00:00<?, ?B/s]

special_tokens_map.json:   0%|          | 0.00/112 [00:00<?, ?B/s]

config.json:   0%|          | 0.00/743 [00:00<?, ?B/s]

model.safetensors:   0%|          | 0.00/438M [00:00<?, ?B/s]

Linear Dim set to: 128 for downcasting

In [50]:

pretty_print(docs)

pretty_print(docs)

Table of Contents
Airbnb, Inc.
Consolidated Statements of Operations
(in millions, except per share amounts)
Year Ended December 31,
2020 2021 2022
Revenue $ 3,378 $ 5,992 $ 8,399 
Costs and expenses:
Cost of revenue 876 1,156 1,499 
Operations and support 878 847 1,041 
Product development 2,753 1,425 1,502 
Sales and marketing 1,175 1,186 1,516 
General and administrative 1,135 836 950 
Restructuring charges 151 113 89 
Total costs and expenses 6,968 5,563 6,597 
Income (loss) from operations (3,590) 429 1,802 
Interest income 27 13 186 
Interest expense (172) (438) (24)
Other income (expense), net (947) (304) 25 
Income (loss) before income taxes (4,682) (300) 1,989 
Provision for (benefit from) income taxes (97) 52 96 
Net income (loss) $ (4,585)$ (352)$ 1,893 
Net income (loss) per share attributable to Class A and Class B common stockholders:
Basic $ (16.12)$ (0.57)$ 2.97 
Diluted $ (16.12)$ (0.57)$ 2.79


In addition, the number of listings on Airbnb may decline as a result of a number of other factors affecting Hosts, including: the COVID-19 pandemic; enforcement or threatenedenforcement of laws and regulations, including short-term occupancy and tax laws; private groups, such as homeowners, landlords, and condominium and neighborhood
associations, adopting and enforcing contracts that prohibit or restrict home sharing; leases, mortgages, and other agreements, or regulations that purport to ban or otherwise restrict
home sharing; Hosts opting for long-term rentals on other third-party platforms as an alternative to listing on our platform; economic, social, and political factors; perceptions of trust
and safety on and off our platform; negative experiences with guests, including guests who damage Host property, throw unauthorized parties, or engage in violent and unlawful


Increased operating expenses, decreased revenue, negative publicity, negative reaction from our Hosts and guests and other stakeholders, or other adverse impacts from any of the
above factors or other risks related to our international operations could materially adversely affect our brand, reputation, business, results of operations, and financial condition.
In addition, we will continue to incur significant expenses to operate our outbound business in China, and we may never achieve profitability in that market. These factors, combined
with sentiment of the workforce in China, and China’s policy towards foreign direct investment may particularly impact our operations in China. In addition, we need to ensure that
our business practices in China are compliant with local laws and regulations, which may be interpreted and enforced in ways that are different from our interpretation, and/or create


Made Possible by Hosts, Strangers, AirCover, Categories, and OMG marketing campaigns and launches, a $67.9 million increase in our search engine marketing and advertising
spend, a $25.1 million increase in payroll-related expenses due to growth in headcount and increase in compensation costs, a $22.0 million increase in third-party service provider
expenses, and a $11.1 million increase in coupon expense in line with increase in revenue and launch of AirCover for guests, partially offset by a decrease of $22.9 million related to
the changes in the fair value of contingent consideration related to a 2019 acquisition.
General and Administrative
2021 2022 % Change
(in millions, except percentages)
General and administrative $ 836 $ 950 14 %
Percentage of revenue 14 % 11 %
General and administrative expense increased $114.0 million, or 14%, in 2022 compared to 2021, primarily due to an increase in other business and operational taxes of $41.3


Our success depends significantly on existing guests continuing to book and attracting new guests to book on our platform. Our ability to attract and retain guests could be materially
adversely affected by a number of factors discussed elsewhere in these “Risk Factors,” including:
• events beyond our control such as the ongoing COVID-19 pandemic, other pandemics and health concerns, restrictions on travel, immigration, trade disputes, economic
downturns, and the impact of climate change on travel including the availability of preferred destinations and the increase in the frequency and severity of weather-relatedevents, including fires, floods, droughts, extreme temperatures and ambient temperature increases, severe weather and other natural disasters, and the impact of other
climate change on seasonal destinations;
• political, social, or economic instability;

Cross Encoder Reranker¶

Uses cross encoder models are rerankers. Uses sentence transformer implementation locally

Latency: 1.38 s ± 64.6 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

In [41]:

from lancedb.rerankers import CrossEncoderReranker

reranker=CrossEncoderReranker()
docs = table.search(query_type="hybrid").vector(query).text(str_query).limit(5).rerank(reranker).to_pandas()["text"].to_list()

from lancedb.rerankers import CrossEncoderReranker reranker=CrossEncoderReranker() docs = table.search(query_type="hybrid").vector(query).text(str_query).limit(5).rerank(reranker).to_pandas()["text"].to_list()

/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_auth.py:94: UserWarning: 
The secret `HF_TOKEN` does not exist in your Colab secrets.
To authenticate with the Hugging Face Hub, create a token in your settings tab (https://huggingface.co/settings/tokens), set it as secret in your Google Colab and restart your session.
You will be able to reuse this secret in all of your notebooks.
Please note that authentication is recommended but still optional to access public models or datasets.
  warnings.warn(

config.json:   0%|          | 0.00/612 [00:00<?, ?B/s]

pytorch_model.bin:   0%|          | 0.00/268M [00:00<?, ?B/s]

tokenizer_config.json:   0%|          | 0.00/541 [00:00<?, ?B/s]

vocab.txt:   0%|          | 0.00/232k [00:00<?, ?B/s]

special_tokens_map.json:   0%|          | 0.00/112 [00:00<?, ?B/s]

model.safetensors:   0%|          | 0.00/268M [00:00<?, ?B/s]

In [42]:

pretty_print(docs)

pretty_print(docs)

Table of Contents
Airbnb, Inc.
Consolidated Statements of Operations
(in millions, except per share amounts)
Year Ended December 31,
2020 2021 2022
Revenue $ 3,378 $ 5,992 $ 8,399 
Costs and expenses:
Cost of revenue 876 1,156 1,499 
Operations and support 878 847 1,041 
Product development 2,753 1,425 1,502 
Sales and marketing 1,175 1,186 1,516 
General and administrative 1,135 836 950 
Restructuring charges 151 113 89 
Total costs and expenses 6,968 5,563 6,597 
Income (loss) from operations (3,590) 429 1,802 
Interest income 27 13 186 
Interest expense (172) (438) (24)
Other income (expense), net (947) (304) 25 
Income (loss) before income taxes (4,682) (300) 1,989 
Provision for (benefit from) income taxes (97) 52 96 
Net income (loss) $ (4,585)$ (352)$ 1,893 
Net income (loss) per share attributable to Class A and Class B common stockholders:
Basic $ (16.12)$ (0.57)$ 2.97 
Diluted $ (16.12)$ (0.57)$ 2.79


Made Possible by Hosts, Strangers, AirCover, Categories, and OMG marketing campaigns and launches, a $67.9 million increase in our search engine marketing and advertising
spend, a $25.1 million increase in payroll-related expenses due to growth in headcount and increase in compensation costs, a $22.0 million increase in third-party service provider
expenses, and a $11.1 million increase in coupon expense in line with increase in revenue and launch of AirCover for guests, partially offset by a decrease of $22.9 million related to
the changes in the fair value of contingent consideration related to a 2019 acquisition.
General and Administrative
2021 2022 % Change
(in millions, except percentages)
General and administrative $ 836 $ 950 14 %
Percentage of revenue 14 % 11 %
General and administrative expense increased $114.0 million, or 14%, in 2022 compared to 2021, primarily due to an increase in other business and operational taxes of $41.3


Increased operating expenses, decreased revenue, negative publicity, negative reaction from our Hosts and guests and other stakeholders, or other adverse impacts from any of the
above factors or other risks related to our international operations could materially adversely affect our brand, reputation, business, results of operations, and financial condition.
In addition, we will continue to incur significant expenses to operate our outbound business in China, and we may never achieve profitability in that market. These factors, combined
with sentiment of the workforce in China, and China’s policy towards foreign direct investment may particularly impact our operations in China. In addition, we need to ensure that
our business practices in China are compliant with local laws and regulations, which may be interpreted and enforced in ways that are different from our interpretation, and/or create


In addition, the number of listings on Airbnb may decline as a result of a number of other factors affecting Hosts, including: the COVID-19 pandemic; enforcement or threatenedenforcement of laws and regulations, including short-term occupancy and tax laws; private groups, such as homeowners, landlords, and condominium and neighborhood
associations, adopting and enforcing contracts that prohibit or restrict home sharing; leases, mortgages, and other agreements, or regulations that purport to ban or otherwise restrict
home sharing; Hosts opting for long-term rentals on other third-party platforms as an alternative to listing on our platform; economic, social, and political factors; perceptions of trust
and safety on and off our platform; negative experiences with guests, including guests who damage Host property, throw unauthorized parties, or engage in violent and unlawful


Our success depends significantly on existing guests continuing to book and attracting new guests to book on our platform. Our ability to attract and retain guests could be materially
adversely affected by a number of factors discussed elsewhere in these “Risk Factors,” including:
• events beyond our control such as the ongoing COVID-19 pandemic, other pandemics and health concerns, restrictions on travel, immigration, trade disputes, economic
downturns, and the impact of climate change on travel including the availability of preferred destinations and the increase in the frequency and severity of weather-relatedevents, including fires, floods, droughts, extreme temperatures and ambient temperature increases, severe weather and other natural disasters, and the impact of other
climate change on seasonal destinations;
• political, social, or economic instability;

(Experimental) OpenAI Reranker¶

This prompts a chat model to rerank results and is not a dedicated reranker model. This should be treated as experimental. You might exceed the token limit so set the search limits based on your token limit. NOTE: It is recommended to use gpt-4-turbo-preview as older models might lead to bad behaviour

Latency: Can take 10s of seconds if using GPT-4 model

In [ ]:

from lancedb.rerankers import OpenaiReranker

reranker=OpenaiReranker(model_name="gpt-4-turbo-preview")
docs = table.search(query_type="hybrid").vector(query).text(str_query).limit(5).rerank(reranker).to_pandas()["text"].to_list()

from lancedb.rerankers import OpenaiReranker reranker=OpenaiReranker(model_name="gpt-4-turbo-preview") docs = table.search(query_type="hybrid").vector(query).text(str_query).limit(5).rerank(reranker).to_pandas()["text"].to_list()

In [ ]:

pretty_print(docs)

pretty_print(docs)

Use your custom Reranker¶

Hybrid search in LanceDB is designed to be very flexible. You can easily plug in your own Re-reranking logic. To do so, you simply need to implement the base Reranker class:

In [44]:

from lancedb.rerankers import Reranker
import pyarrow as pa

class MyCustomReranker(Reranker):
    def rerank_hybrid(self, query: str, vector_results: pa.Table, fts_results: pa.Table)-> pa.Table:
        combined_results = self.merge(vector_results, fts_results) # Or custom merge algo
        # Custom Reranking logic here

        return combined_results

from lancedb.rerankers import Reranker import pyarrow as pa class MyCustomReranker(Reranker): def rerank_hybrid(self, query: str, vector_results: pa.Table, fts_results: pa.Table)-> pa.Table: combined_results = self.merge(vector_results, fts_results) # Or custom merge algo # Custom Reranking logic here return combined_results

Custom Reranker based on CohereReranker¶

For the sake of simplicity let's build a custom reranker that enhances the Cohere Reranker by accepting a filter query, and accepts other CohereReranker params as kwargs.

For this toy example let's say we want to get rid of docs that represent a table of contents or appendix, as these are semantically close to representing costs but don't represent the specific reasons why operating costs were high.

In [45]:

from typing import List, Union
import pandas as pd
from lancedb.rerankers import CohereReranker

class MofidifiedCohereReranker(CohereReranker):
    def __init__(self, filters: Union[str, List[str]], **kwargs):
        super().__init__(**kwargs)
        filters = filters if isinstance(filters, list) else [filters]
        self.filters = filters

    def rerank_hybrid(self, query: str, vector_results: pa.Table, fts_results: pa.Table)-> pa.Table:
        combined_result = super().rerank_hybrid(query, vector_results, fts_results)
        df = combined_result.to_pandas()
        for filter in self.filters:
            df = df.query("not text.str.contains(@filter)")

        return pa.Table.from_pandas(df)

reranker = MofidifiedCohereReranker(filters="Table of Contents")

from typing import List, Union import pandas as pd from lancedb.rerankers import CohereReranker class MofidifiedCohereReranker(CohereReranker): def __init__(self, filters: Union[str, List[str]], **kwargs): super().__init__(**kwargs) filters = filters if isinstance(filters, list) else [filters] self.filters = filters def rerank_hybrid(self, query: str, vector_results: pa.Table, fts_results: pa.Table)-> pa.Table: combined_result = super().rerank_hybrid(query, vector_results, fts_results) df = combined_result.to_pandas() for filter in self.filters: df = df.query("not text.str.contains(@filter)") return pa.Table.from_pandas(df) reranker = MofidifiedCohereReranker(filters="Table of Contents")

In [46]:

docs = table.search(query_type="hybrid").vector(query).text(str_query).limit(5).rerank(reranker).to_pandas()["text"].to_list()

docs = table.search(query_type="hybrid").vector(query).text(str_query).limit(5).rerank(reranker).to_pandas()["text"].to_list()

In [47]:

pretty_print(docs)

pretty_print(docs)

Increased operating expenses, decreased revenue, negative publicity, negative reaction from our Hosts and guests and other stakeholders, or other adverse impacts from any of the
above factors or other risks related to our international operations could materially adversely affect our brand, reputation, business, results of operations, and financial condition.
In addition, we will continue to incur significant expenses to operate our outbound business in China, and we may never achieve profitability in that market. These factors, combined
with sentiment of the workforce in China, and China’s policy towards foreign direct investment may particularly impact our operations in China. In addition, we need to ensure that
our business practices in China are compliant with local laws and regulations, which may be interpreted and enforced in ways that are different from our interpretation, and/or create


Made Possible by Hosts, Strangers, AirCover, Categories, and OMG marketing campaigns and launches, a $67.9 million increase in our search engine marketing and advertising
spend, a $25.1 million increase in payroll-related expenses due to growth in headcount and increase in compensation costs, a $22.0 million increase in third-party service provider
expenses, and a $11.1 million increase in coupon expense in line with increase in revenue and launch of AirCover for guests, partially offset by a decrease of $22.9 million related to
the changes in the fair value of contingent consideration related to a 2019 acquisition.
General and Administrative
2021 2022 % Change
(in millions, except percentages)
General and administrative $ 836 $ 950 14 %
Percentage of revenue 14 % 11 %
General and administrative expense increased $114.0 million, or 14%, in 2022 compared to 2021, primarily due to an increase in other business and operational taxes of $41.3


• Hosts failing to meet guests’ expectations, including increased expectations for cleanliness in light of the COVID-19 pandemic;• increased competition and use of our competitors’ platforms and services;
• Hosts failing to provide differentiated, high-quality, and an adequate supply of stays or experiences at competitive prices;
• guests not receiving timely and adequate community support from us;
• our failure to provide new or enhanced offerings, tiers, or features that guests value;
• declines or inefficiencies in our marketing efforts;• negative associations with, or reduced awareness of, our brand;
• actual or perceived discrimination by Hosts in deciding whether to accept a requested reservation;
• negative perceptions of the trust and safety on our platform; and
• macroeconomic and other conditions outside of our control affecting travel and hospitality industries generally.


Our success depends significantly on existing guests continuing to book and attracting new guests to book on our platform. Our ability to attract and retain guests could be materially
adversely affected by a number of factors discussed elsewhere in these “Risk Factors,” including:
• events beyond our control such as the ongoing COVID-19 pandemic, other pandemics and health concerns, restrictions on travel, immigration, trade disputes, economic
downturns, and the impact of climate change on travel including the availability of preferred destinations and the increase in the frequency and severity of weather-relatedevents, including fires, floods, droughts, extreme temperatures and ambient temperature increases, severe weather and other natural disasters, and the impact of other
climate change on seasonal destinations;
• political, social, or economic instability;


In addition, the number of listings on Airbnb may decline as a result of a number of other factors affecting Hosts, including: the COVID-19 pandemic; enforcement or threatenedenforcement of laws and regulations, including short-term occupancy and tax laws; private groups, such as homeowners, landlords, and condominium and neighborhood
associations, adopting and enforcing contracts that prohibit or restrict home sharing; leases, mortgages, and other agreements, or regulations that purport to ban or otherwise restrict
home sharing; Hosts opting for long-term rentals on other third-party platforms as an alternative to listing on our platform; economic, social, and political factors; perceptions of trust
and safety on and off our platform; negative experiences with guests, including guests who damage Host property, throw unauthorized parties, or engage in violent and unlawful

As you can see, the document containing the table of contents no longer shows up.