fetch_k – Number of Documents to fetch to pass to MMR algorithm.
lambda_mult – Number between 0 and 1 that determines the degree
of diversity among the results with 0 corresponding
to maximum diversity and 1 to minimum diversity.
Defaults to 0.5.
Returns
List of Documents selected by maximal marginal relevance.
max_marginal_relevance_search_by_vector(embedding: List[float], k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, **kwargs: Any) → List[Document]¶
Return docs selected using the maximal marginal relevance.
Maximal marginal relevance optimizes for similarity to query AND diversity
among selected documents.
Parameters
embedding – Embedding to look up documents similar to.
k – Number of Documents to return. Defaults to 4.
fetch_k – Number of Documents to fetch to pass to MMR algorithm.
lambda_mult – Number between 0 and 1 that determines the degree
of diversity among the results with 0 corresponding
to maximum diversity and 1 to minimum diversity.
Defaults to 0.5.
Returns
List of Documents selected by maximal marginal relevance.
search(query: str, search_type: str, **kwargs: Any) → List[Document]¶
Return docs most similar to query using specified search type.
similarity_search(query: str, k: int = 4, where_str: Optional[str] = None, **kwargs: Any) → List[Document][source]¶
Perform a similarity search with StarRocks
Parameters
query (str) – query string
k (int, optional) – Top K neighbors to retrieve. Defaults to 4.
where_str (Optional[str], optional) – where condition string.
Defaults to None.
NOTE – Please do not let end-user to fill this and always be aware


NOTE – Please do not let end-user to fill this and always be aware
of SQL injection. When dealing with metadatas, remember to
use {self.metadata_column}.attribute instead of attribute
alone. The default name for it is metadata.
Returns
List of Documents
Return type
List[Document]
similarity_search_by_vector(embedding: List[float], k: int = 4, where_str: Optional[str] = None, **kwargs: Any) → List[Document][source]¶
Perform a similarity search with StarRocks by vectors
Parameters
query (str) – query string
k (int, optional) – Top K neighbors to retrieve. Defaults to 4.
where_str (Optional[str], optional) – where condition string.
Defaults to None.
NOTE – Please do not let end-user to fill this and always be aware
of SQL injection. When dealing with metadatas, remember to
use {self.metadata_column}.attribute instead of attribute
alone. The default name for it is metadata.
Returns
List of (Document, similarity)
Return type
List[Document]
similarity_search_with_relevance_scores(query: str, k: int = 4, where_str: Optional[str] = None, **kwargs: Any) → List[Tuple[Document, float]][source]¶
Perform a similarity search with StarRocks
Parameters
query (str) – query string
k (int, optional) – Top K neighbors to retrieve. Defaults to 4.
where_str (Optional[str], optional) – where condition string.
Defaults to None.
NOTE – Please do not let end-user to fill this and always be aware
of SQL injection. When dealing with metadatas, remember to
use {self.metadata_column}.attribute instead of attribute
alone. The default name for it is metadata.
Returns
List of documents
Return type


alone. The default name for it is metadata.
Returns
List of documents
Return type
List[Document]
similarity_search_with_score(*args: Any, **kwargs: Any) → List[Tuple[Document, float]]¶
Run similarity search with distance.
Examples using StarRocks¶
StarRocks


langchain.vectorstores.chroma.Chroma¶
class langchain.vectorstores.chroma.Chroma(collection_name: str = 'langchain', embedding_function: Optional[Embeddings] = None, persist_directory: Optional[str] = None, client_settings: Optional[chromadb.config.Settings] = None, collection_metadata: Optional[Dict] = None, client: Optional[chromadb.Client] = None, relevance_score_fn: Optional[Callable[[float], float]] = None)[source]¶
ChromaDB vector store.
To use, you should have the chromadb python package installed.
Example
from langchain.vectorstores import Chroma
from langchain.embeddings.openai import OpenAIEmbeddings
embeddings = OpenAIEmbeddings()
vectorstore = Chroma("langchain_store", embeddings)
Initialize with a Chroma client.
Attributes
embeddings
Access the query embedding object if available.
Methods
__init__([collection_name, ...])
Initialize with a Chroma client.
aadd_documents(documents, **kwargs)
Run more documents through the embeddings and add to the vectorstore.
aadd_texts(texts[, metadatas])
Run more texts through the embeddings and add to the vectorstore.
add_documents(documents, **kwargs)
Run more documents through the embeddings and add to the vectorstore.
add_images(uris[, metadatas, ids])
Run more images through the embeddings and add to the vectorstore.
add_texts(texts[, metadatas, ids])
Run more texts through the embeddings and add to the vectorstore.
adelete([ids])
Delete by vector ID or other criteria.
afrom_documents(documents, embedding, **kwargs)
Return VectorStore initialized from documents and embeddings.
afrom_texts(texts, embedding[, metadatas])


afrom_texts(texts, embedding[, metadatas])
Return VectorStore initialized from texts and embeddings.
amax_marginal_relevance_search(query[, k, ...])
Return docs selected using the maximal marginal relevance.
amax_marginal_relevance_search_by_vector(...)
Return docs selected using the maximal marginal relevance.
as_retriever(**kwargs)
Return VectorStoreRetriever initialized from this VectorStore.
asearch(query, search_type, **kwargs)
Return docs most similar to query using specified search type.
asimilarity_search(query[, k])
Return docs most similar to query.
asimilarity_search_by_vector(embedding[, k])
Return docs most similar to embedding vector.
asimilarity_search_with_relevance_scores(query)
Return docs and relevance scores in the range [0, 1], asynchronously.
asimilarity_search_with_score(*args, **kwargs)
Run similarity search with distance asynchronously.
delete([ids])
Delete by vector IDs.
delete_collection()
Delete the collection.
encode_image(uri)
Get base64 string from image URI.
from_documents(documents[, embedding, ids, ...])
Create a Chroma vectorstore from a list of documents.
from_texts(texts[, embedding, metadatas, ...])
Create a Chroma vectorstore from a raw documents.
get([ids, where, limit, offset, ...])
Gets the collection.
max_marginal_relevance_search(query[, k, ...])
Return docs selected using the maximal marginal relevance.
max_marginal_relevance_search_by_vector(...)
Return docs selected using the maximal marginal relevance.
persist()
Persist the collection.
search(query, search_type, **kwargs)
Return docs most similar to query using specified search type.


Return docs most similar to query using specified search type.
similarity_search(query[, k, filter])
Run similarity search with Chroma.
similarity_search_by_vector(embedding[, k, ...])
Return docs most similar to embedding vector.
similarity_search_by_vector_with_relevance_scores(...)
Return docs most similar to embedding vector and similarity score.
similarity_search_with_relevance_scores(query)
Return docs and relevance scores in the range [0, 1].
similarity_search_with_score(query[, k, ...])
Run similarity search with Chroma with distance.
update_document(document_id, document)
Update a document in the collection.
update_documents(ids, documents)
Update a document in the collection.
__init__(collection_name: str = 'langchain', embedding_function: Optional[Embeddings] = None, persist_directory: Optional[str] = None, client_settings: Optional[chromadb.config.Settings] = None, collection_metadata: Optional[Dict] = None, client: Optional[chromadb.Client] = None, relevance_score_fn: Optional[Callable[[float], float]] = None) → None[source]¶
Initialize with a Chroma client.
async aadd_documents(documents: List[Document], **kwargs: Any) → List[str]¶
Run more documents through the embeddings and add to the vectorstore.
Parameters
(List[Document] (documents) – Documents to add to the vectorstore.
Returns
List of IDs of the added texts.
Return type
List[str]
async aadd_texts(texts: Iterable[str], metadatas: Optional[List[dict]] = None, **kwargs: Any) → List[str]¶
Run more texts through the embeddings and add to the vectorstore.
add_documents(documents: List[Document], **kwargs: Any) → List[str]¶


add_documents(documents: List[Document], **kwargs: Any) → List[str]¶
Run more documents through the embeddings and add to the vectorstore.
Parameters
(List[Document] (documents) – Documents to add to the vectorstore.
Returns
List of IDs of the added texts.
Return type
List[str]
add_images(uris: List[str], metadatas: Optional[List[dict]] = None, ids: Optional[List[str]] = None, **kwargs: Any) → List[str][source]¶
Run more images through the embeddings and add to the vectorstore.
Parameters
images (List[List[float]]) – Images to add to the vectorstore.
metadatas (Optional[List[dict]], optional) – Optional list of metadatas.
ids (Optional[List[str]], optional) – Optional list of IDs.
Returns
List of IDs of the added images.
Return type
List[str]
add_texts(texts: Iterable[str], metadatas: Optional[List[dict]] = None, ids: Optional[List[str]] = None, **kwargs: Any) → List[str][source]¶
Run more texts through the embeddings and add to the vectorstore.
Parameters
texts (Iterable[str]) – Texts to add to the vectorstore.
metadatas (Optional[List[dict]], optional) – Optional list of metadatas.
ids (Optional[List[str]], optional) – Optional list of IDs.
Returns
List of IDs of the added texts.
Return type
List[str]
async adelete(ids: Optional[List[str]] = None, **kwargs: Any) → Optional[bool]¶
Delete by vector ID or other criteria.
Parameters
ids – List of ids to delete.
**kwargs – Other keyword arguments that subclasses might use.
Returns
True if deletion is successful,
False otherwise, None if not implemented.


Returns
True if deletion is successful,
False otherwise, None if not implemented.
Return type
Optional[bool]
async classmethod afrom_documents(documents: List[Document], embedding: Embeddings, **kwargs: Any) → VST¶
Return VectorStore initialized from documents and embeddings.
async classmethod afrom_texts(texts: List[str], embedding: Embeddings, metadatas: Optional[List[dict]] = None, **kwargs: Any) → VST¶
Return VectorStore initialized from texts and embeddings.
async amax_marginal_relevance_search(query: str, k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, **kwargs: Any) → List[Document]¶
Return docs selected using the maximal marginal relevance.
async amax_marginal_relevance_search_by_vector(embedding: List[float], k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, **kwargs: Any) → List[Document]¶
Return docs selected using the maximal marginal relevance.
as_retriever(**kwargs: Any) → VectorStoreRetriever¶
Return VectorStoreRetriever initialized from this VectorStore.
Parameters
search_type (Optional[str]) – Defines the type of search that
the Retriever should perform.
Can be “similarity” (default), “mmr”, or
“similarity_score_threshold”.
search_kwargs (Optional[Dict]) – Keyword arguments to pass to the
search function. Can include things like:
k: Amount of documents to return (Default: 4)
score_threshold: Minimum relevance threshold
for similarity_score_threshold
fetch_k: Amount of documents to pass to MMR algorithm (Default: 20)
lambda_mult: Diversity of results returned by MMR;


lambda_mult: Diversity of results returned by MMR;
1 for minimum diversity and 0 for maximum. (Default: 0.5)
filter: Filter by document metadata
Returns
Retriever class for VectorStore.
Return type
VectorStoreRetriever
Examples:
# Retrieve more documents with higher diversity
# Useful if your dataset has many similar documents
docsearch.as_retriever(
    search_type="mmr",
    search_kwargs={'k': 6, 'lambda_mult': 0.25}
)
# Fetch more documents for the MMR algorithm to consider
# But only return the top 5
docsearch.as_retriever(
    search_type="mmr",
    search_kwargs={'k': 5, 'fetch_k': 50}
)
# Only retrieve documents that have a relevance score
# Above a certain threshold
docsearch.as_retriever(
    search_type="similarity_score_threshold",
    search_kwargs={'score_threshold': 0.8}
)
# Only get the single most similar document from the dataset
docsearch.as_retriever(search_kwargs={'k': 1})
# Use a filter to only retrieve documents from a specific paper
docsearch.as_retriever(
    search_kwargs={'filter': {'paper_title':'GPT-4 Technical Report'}}
)
async asearch(query: str, search_type: str, **kwargs: Any) → List[Document]¶
Return docs most similar to query using specified search type.
async asimilarity_search(query: str, k: int = 4, **kwargs: Any) → List[Document]¶
Return docs most similar to query.
async asimilarity_search_by_vector(embedding: List[float], k: int = 4, **kwargs: Any) → List[Document]¶


Return docs most similar to embedding vector.
async asimilarity_search_with_relevance_scores(query: str, k: int = 4, **kwargs: Any) → List[Tuple[Document, float]]¶
Return docs and relevance scores in the range [0, 1], asynchronously.
0 is dissimilar, 1 is most similar.
Parameters
query – input text
k – Number of Documents to return. Defaults to 4.
**kwargs – kwargs to be passed to similarity search. Should include:
score_threshold: Optional, a floating point value between 0 to 1 to
filter the resulting set of retrieved docs
Returns
List of Tuples of (doc, similarity_score)
async asimilarity_search_with_score(*args: Any, **kwargs: Any) → List[Tuple[Document, float]]¶
Run similarity search with distance asynchronously.
delete(ids: Optional[List[str]] = None, **kwargs: Any) → None[source]¶
Delete by vector IDs.
Parameters
ids – List of ids to delete.
delete_collection() → None[source]¶
Delete the collection.
encode_image(uri: str) → str[source]¶
Get base64 string from image URI.
classmethod from_documents(documents: List[Document], embedding: Optional[Embeddings] = None, ids: Optional[List[str]] = None, collection_name: str = 'langchain', persist_directory: Optional[str] = None, client_settings: Optional[chromadb.config.Settings] = None, client: Optional[chromadb.Client] = None, collection_metadata: Optional[Dict] = None, **kwargs: Any) → Chroma[source]¶
Create a Chroma vectorstore from a list of documents.
If a persist_directory is specified, the collection will be persisted there.
Otherwise, the data will be ephemeral in-memory.
Parameters


Otherwise, the data will be ephemeral in-memory.
Parameters
collection_name (str) – Name of the collection to create.
persist_directory (Optional[str]) – Directory to persist the collection.
ids (Optional[List[str]]) – List of document IDs. Defaults to None.
documents (List[Document]) – List of documents to add to the vectorstore.
embedding (Optional[Embeddings]) – Embedding function. Defaults to None.
client_settings (Optional[chromadb.config.Settings]) – Chroma client settings
collection_metadata (Optional[Dict]) – Collection configurations.
Defaults to None.
Returns
Chroma vectorstore.
Return type
Chroma
classmethod from_texts(texts: List[str], embedding: Optional[Embeddings] = None, metadatas: Optional[List[dict]] = None, ids: Optional[List[str]] = None, collection_name: str = 'langchain', persist_directory: Optional[str] = None, client_settings: Optional[chromadb.config.Settings] = None, client: Optional[chromadb.Client] = None, collection_metadata: Optional[Dict] = None, **kwargs: Any) → Chroma[source]¶
Create a Chroma vectorstore from a raw documents.
If a persist_directory is specified, the collection will be persisted there.
Otherwise, the data will be ephemeral in-memory.
Parameters
texts (List[str]) – List of texts to add to the collection.
collection_name (str) – Name of the collection to create.
persist_directory (Optional[str]) – Directory to persist the collection.
embedding (Optional[Embeddings]) – Embedding function. Defaults to None.
metadatas (Optional[List[dict]]) – List of metadatas. Defaults to None.
ids (Optional[List[str]]) – List of document IDs. Defaults to None.


ids (Optional[List[str]]) – List of document IDs. Defaults to None.
client_settings (Optional[chromadb.config.Settings]) – Chroma client settings
collection_metadata (Optional[Dict]) – Collection configurations.
Defaults to None.
Returns
Chroma vectorstore.
Return type
Chroma
get(ids: Optional[OneOrMany[ID]] = None, where: Optional[Where] = None, limit: Optional[int] = None, offset: Optional[int] = None, where_document: Optional[WhereDocument] = None, include: Optional[List[str]] = None) → Dict[str, Any][source]¶
Gets the collection.
Parameters
ids – The ids of the embeddings to get. Optional.
where – A Where type dict used to filter results by.
E.g. {“color” : “red”, “price”: 4.20}. Optional.
limit – The number of documents to return. Optional.
offset – The offset to start returning results from.
Useful for paging results with limit. Optional.
where_document – A WhereDocument type dict used to filter by the documents.
E.g. {$contains: “hello”}. Optional.
include – A list of what to include in the results.
Can contain “embeddings”, “metadatas”, “documents”.
Ids are always included.
Defaults to [“metadatas”, “documents”]. Optional.
max_marginal_relevance_search(query: str, k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, filter: Optional[Dict[str, str]] = None, where_document: Optional[Dict[str, str]] = None, **kwargs: Any) → List[Document][source]¶
Return docs selected using the maximal marginal relevance.
Maximal marginal relevance optimizes for similarity to query AND diversity


Maximal marginal relevance optimizes for similarity to query AND diversity
among selected documents.
Parameters
query – Text to look up documents similar to.
k – Number of Documents to return. Defaults to 4.
fetch_k – Number of Documents to fetch to pass to MMR algorithm.
lambda_mult – Number between 0 and 1 that determines the degree
of diversity among the results with 0 corresponding
to maximum diversity and 1 to minimum diversity.
Defaults to 0.5.
filter (Optional[Dict[str, str]]) – Filter by metadata. Defaults to None.
Returns
List of Documents selected by maximal marginal relevance.
max_marginal_relevance_search_by_vector(embedding: List[float], k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, filter: Optional[Dict[str, str]] = None, where_document: Optional[Dict[str, str]] = None, **kwargs: Any) → List[Document][source]¶
Return docs selected using the maximal marginal relevance.
Maximal marginal relevance optimizes for similarity to query AND diversity
among selected documents.
Parameters
embedding – Embedding to look up documents similar to.
k – Number of Documents to return. Defaults to 4.
fetch_k – Number of Documents to fetch to pass to MMR algorithm.
lambda_mult – Number between 0 and 1 that determines the degree
of diversity among the results with 0 corresponding
to maximum diversity and 1 to minimum diversity.
Defaults to 0.5.
filter (Optional[Dict[str, str]]) – Filter by metadata. Defaults to None.
Returns
List of Documents selected by maximal marginal relevance.
persist() → None[source]¶
Persist the collection.
This can be used to explicitly persist the data to disk.


Persist the collection.
This can be used to explicitly persist the data to disk.
It will also be called automatically when the object is destroyed.
search(query: str, search_type: str, **kwargs: Any) → List[Document]¶
Return docs most similar to query using specified search type.
similarity_search(query: str, k: int = 4, filter: Optional[Dict[str, str]] = None, **kwargs: Any) → List[Document][source]¶
Run similarity search with Chroma.
Parameters
query (str) – Query text to search for.
k (int) – Number of results to return. Defaults to 4.
filter (Optional[Dict[str, str]]) – Filter by metadata. Defaults to None.
Returns
List of documents most similar to the query text.
Return type
List[Document]
similarity_search_by_vector(embedding: List[float], k: int = 4, filter: Optional[Dict[str, str]] = None, where_document: Optional[Dict[str, str]] = None, **kwargs: Any) → List[Document][source]¶
Return docs most similar to embedding vector.
:param embedding: Embedding to look up documents similar to.
:type embedding: List[float]
:param k: Number of Documents to return. Defaults to 4.
:type k: int
:param filter: Filter by metadata. Defaults to None.
:type filter: Optional[Dict[str, str]]
Returns
List of Documents most similar to the query vector.
similarity_search_by_vector_with_relevance_scores(embedding: List[float], k: int = 4, filter: Optional[Dict[str, str]] = None, where_document: Optional[Dict[str, str]] = None, **kwargs: Any) → List[Tuple[Document, float]][source]¶


Return docs most similar to embedding vector and similarity score.
Parameters
embedding (List[float]) – Embedding to look up documents similar to.
k (int) – Number of Documents to return. Defaults to 4.
filter (Optional[Dict[str, str]]) – Filter by metadata. Defaults to None.
Returns
List of documents most similar to
the query text and cosine distance in float for each.
Lower score represents more similarity.
Return type
List[Tuple[Document, float]]
similarity_search_with_relevance_scores(query: str, k: int = 4, **kwargs: Any) → List[Tuple[Document, float]]¶
Return docs and relevance scores in the range [0, 1].
0 is dissimilar, 1 is most similar.
Parameters
query – input text
k – Number of Documents to return. Defaults to 4.
**kwargs – kwargs to be passed to similarity search. Should include:
score_threshold: Optional, a floating point value between 0 to 1 to
filter the resulting set of retrieved docs
Returns
List of Tuples of (doc, similarity_score)
similarity_search_with_score(query: str, k: int = 4, filter: Optional[Dict[str, str]] = None, where_document: Optional[Dict[str, str]] = None, **kwargs: Any) → List[Tuple[Document, float]][source]¶
Run similarity search with Chroma with distance.
Parameters
query (str) – Query text to search for.
k (int) – Number of results to return. Defaults to 4.
filter (Optional[Dict[str, str]]) – Filter by metadata. Defaults to None.
Returns
List of documents most similar to
the query text and cosine distance in float for each.
Lower score represents more similarity.
Return type


Lower score represents more similarity.
Return type
List[Tuple[Document, float]]
update_document(document_id: str, document: Document) → None[source]¶
Update a document in the collection.
Parameters
document_id (str) – ID of the document to update.
document (Document) – Document to update.
update_documents(ids: List[str], documents: List[Document]) → None[source]¶
Update a document in the collection.
Parameters
ids (List[str]) – List of ids of the document to update.
documents (List[Document]) – List of documents to update.
Examples using Chroma¶
RePhraseQueryRetriever
LOTR (Merger Retriever)
Ollama
Confident
Chroma
Vectorstore
StarRocks
Psychic
Docugami
Set env var OPENAI_API_KEY or load from a .env file:
Set env var OPENAI_API_KEY or load from a .env file
Question Answering
Perform context-aware text splitting
Use local LLMs
Retrieve from vector stores directly
Improve document indexing with HyDE
Structure answers with OpenAI functions
SalesGPT - Your Context-Aware AI Sales Assistant With Knowledge Base
MultiVector Retriever
MultiQueryRetriever
Parent Document Retriever
WebResearchRetriever
Lost in the middle: The problem with long contexts
Memory in the Multi-Input Chain
Combine agents and vector stores
Few-shot examples for chat models
Router
Loading from LangChainHub


langchain.vectorstores.scann.dependable_scann_import¶
langchain.vectorstores.scann.dependable_scann_import() → Any[source]¶
Import scann if available, otherwise raise error.


langchain.vectorstores.azure_cosmos_db.AzureCosmosDBVectorSearch¶
class langchain.vectorstores.azure_cosmos_db.AzureCosmosDBVectorSearch(collection: Collection[CosmosDBDocumentType], embedding: Embeddings, *, index_name: str = 'vectorSearchIndex', text_key: str = 'textContent', embedding_key: str = 'vectorContent')[source]¶
Azure Cosmos DB for MongoDB vCore vector store.
To use, you should have both:
- the pymongo python package installed
- a connection string associated with a MongoDB VCore Cluster
Example
. code-block:: python
from langchain.vectorstores import AzureCosmosDBVectorSearch
from langchain.embeddings.openai import OpenAIEmbeddings
from pymongo import MongoClient
mongo_client = MongoClient(“<YOUR-CONNECTION-STRING>”)
collection = mongo_client[“<db_name>”][“<collection_name>”]
embeddings = OpenAIEmbeddings()
vectorstore = AzureCosmosDBVectorSearch(collection, embeddings)
Constructor for AzureCosmosDBVectorSearch
Parameters
collection – MongoDB collection to add the texts to.
embedding – Text embedding model to use.
index_name – Name of the Atlas Search index.
text_key – MongoDB field that will contain the text
for each document.
embedding_key – MongoDB field that will contain the embedding
for each document.
Attributes
embeddings
Access the query embedding object if available.
Methods
__init__(collection, embedding, *[, ...])
Constructor for AzureCosmosDBVectorSearch
aadd_documents(documents, **kwargs)
Run more documents through the embeddings and add to the vectorstore.
aadd_texts(texts[, metadatas])
Run more texts through the embeddings and add to the vectorstore.
add_documents(documents, **kwargs)


add_documents(documents, **kwargs)
Run more documents through the embeddings and add to the vectorstore.
add_texts(texts[, metadatas])
Run more texts through the embeddings and add to the vectorstore.
adelete([ids])
Delete by vector ID or other criteria.
afrom_documents(documents, embedding, **kwargs)
Return VectorStore initialized from documents and embeddings.
afrom_texts(texts, embedding[, metadatas])
Return VectorStore initialized from texts and embeddings.
amax_marginal_relevance_search(query[, k, ...])
Return docs selected using the maximal marginal relevance.
amax_marginal_relevance_search_by_vector(...)
Return docs selected using the maximal marginal relevance.
as_retriever(**kwargs)
Return VectorStoreRetriever initialized from this VectorStore.
asearch(query, search_type, **kwargs)
Return docs most similar to query using specified search type.
asimilarity_search(query[, k])
Return docs most similar to query.
asimilarity_search_by_vector(embedding[, k])
Return docs most similar to embedding vector.
asimilarity_search_with_relevance_scores(query)
Return docs and relevance scores in the range [0, 1], asynchronously.
asimilarity_search_with_score(*args, **kwargs)
Run similarity search with distance asynchronously.
create_index([num_lists, dimensions, similarity])
Creates an index using the index name specified at
delete([ids])
Delete by vector ID or other criteria.
delete_document_by_id([document_id])
Removes a Specific Document by Id
delete_index()
Deletes the index specified during instance construction if it exists
from_connection_string(connection_string, ...)
Creates an Instance of AzureCosmosDBVectorSearch from a Connection String


Creates an Instance of AzureCosmosDBVectorSearch from a Connection String
from_documents(documents, embedding, **kwargs)
Return VectorStore initialized from documents and embeddings.
from_texts(texts, embedding[, metadatas, ...])
Return VectorStore initialized from texts and embeddings.
get_index_name()
Returns the index name
index_exists()
Verifies if the specified index name during instance
max_marginal_relevance_search(query[, k, ...])
Return docs selected using the maximal marginal relevance.
max_marginal_relevance_search_by_vector(...)
Return docs selected using the maximal marginal relevance.
search(query, search_type, **kwargs)
Return docs most similar to query using specified search type.
similarity_search(query[, k])
Return docs most similar to query.
similarity_search_by_vector(embedding[, k])
Return docs most similar to embedding vector.
similarity_search_with_relevance_scores(query)
Return docs and relevance scores in the range [0, 1].
similarity_search_with_score(query[, k])
Run similarity search with distance.
__init__(collection: Collection[CosmosDBDocumentType], embedding: Embeddings, *, index_name: str = 'vectorSearchIndex', text_key: str = 'textContent', embedding_key: str = 'vectorContent')[source]¶
Constructor for AzureCosmosDBVectorSearch
Parameters
collection – MongoDB collection to add the texts to.
embedding – Text embedding model to use.
index_name – Name of the Atlas Search index.
text_key – MongoDB field that will contain the text
for each document.
embedding_key – MongoDB field that will contain the embedding
for each document.
async aadd_documents(documents: List[Document], **kwargs: Any) → List[str]¶


Run more documents through the embeddings and add to the vectorstore.
Parameters
(List[Document] (documents) – Documents to add to the vectorstore.
Returns
List of IDs of the added texts.
Return type
List[str]
async aadd_texts(texts: Iterable[str], metadatas: Optional[List[dict]] = None, **kwargs: Any) → List[str]¶
Run more texts through the embeddings and add to the vectorstore.
add_documents(documents: List[Document], **kwargs: Any) → List[str]¶
Run more documents through the embeddings and add to the vectorstore.
Parameters
(List[Document] (documents) – Documents to add to the vectorstore.
Returns
List of IDs of the added texts.
Return type
List[str]
add_texts(texts: Iterable[str], metadatas: Optional[List[Dict[str, Any]]] = None, **kwargs: Any) → List[source]¶
Run more texts through the embeddings and add to the vectorstore.
Parameters
texts – Iterable of strings to add to the vectorstore.
metadatas – Optional list of metadatas associated with the texts.
kwargs – vectorstore specific parameters
Returns
List of ids from adding the texts into the vectorstore.
async adelete(ids: Optional[List[str]] = None, **kwargs: Any) → Optional[bool]¶
Delete by vector ID or other criteria.
Parameters
ids – List of ids to delete.
**kwargs – Other keyword arguments that subclasses might use.
Returns
True if deletion is successful,
False otherwise, None if not implemented.
Return type
Optional[bool]
async classmethod afrom_documents(documents: List[Document], embedding: Embeddings, **kwargs: Any) → VST¶
Return VectorStore initialized from documents and embeddings.


Return VectorStore initialized from documents and embeddings.
async classmethod afrom_texts(texts: List[str], embedding: Embeddings, metadatas: Optional[List[dict]] = None, **kwargs: Any) → VST¶
Return VectorStore initialized from texts and embeddings.
async amax_marginal_relevance_search(query: str, k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, **kwargs: Any) → List[Document]¶
Return docs selected using the maximal marginal relevance.
async amax_marginal_relevance_search_by_vector(embedding: List[float], k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, **kwargs: Any) → List[Document]¶
Return docs selected using the maximal marginal relevance.
as_retriever(**kwargs: Any) → VectorStoreRetriever¶
Return VectorStoreRetriever initialized from this VectorStore.
Parameters
search_type (Optional[str]) – Defines the type of search that
the Retriever should perform.
Can be “similarity” (default), “mmr”, or
“similarity_score_threshold”.
search_kwargs (Optional[Dict]) – Keyword arguments to pass to the
search function. Can include things like:
k: Amount of documents to return (Default: 4)
score_threshold: Minimum relevance threshold
for similarity_score_threshold
fetch_k: Amount of documents to pass to MMR algorithm (Default: 20)
lambda_mult: Diversity of results returned by MMR;
1 for minimum diversity and 0 for maximum. (Default: 0.5)
filter: Filter by document metadata
Returns
Retriever class for VectorStore.
Return type
VectorStoreRetriever
Examples:
# Retrieve more documents with higher diversity


Return type
VectorStoreRetriever
Examples:
# Retrieve more documents with higher diversity
# Useful if your dataset has many similar documents
docsearch.as_retriever(
    search_type="mmr",
    search_kwargs={'k': 6, 'lambda_mult': 0.25}
)
# Fetch more documents for the MMR algorithm to consider
# But only return the top 5
docsearch.as_retriever(
    search_type="mmr",
    search_kwargs={'k': 5, 'fetch_k': 50}
)
# Only retrieve documents that have a relevance score
# Above a certain threshold
docsearch.as_retriever(
    search_type="similarity_score_threshold",
    search_kwargs={'score_threshold': 0.8}
)
# Only get the single most similar document from the dataset
docsearch.as_retriever(search_kwargs={'k': 1})
# Use a filter to only retrieve documents from a specific paper
docsearch.as_retriever(
    search_kwargs={'filter': {'paper_title':'GPT-4 Technical Report'}}
)
async asearch(query: str, search_type: str, **kwargs: Any) → List[Document]¶
Return docs most similar to query using specified search type.
async asimilarity_search(query: str, k: int = 4, **kwargs: Any) → List[Document]¶
Return docs most similar to query.
async asimilarity_search_by_vector(embedding: List[float], k: int = 4, **kwargs: Any) → List[Document]¶
Return docs most similar to embedding vector.
async asimilarity_search_with_relevance_scores(query: str, k: int = 4, **kwargs: Any) → List[Tuple[Document, float]]¶


Return docs and relevance scores in the range [0, 1], asynchronously.
0 is dissimilar, 1 is most similar.
Parameters
query – input text
k – Number of Documents to return. Defaults to 4.
**kwargs – kwargs to be passed to similarity search. Should include:
score_threshold: Optional, a floating point value between 0 to 1 to
filter the resulting set of retrieved docs
Returns
List of Tuples of (doc, similarity_score)
async asimilarity_search_with_score(*args: Any, **kwargs: Any) → List[Tuple[Document, float]]¶
Run similarity search with distance asynchronously.
create_index(num_lists: int = 100, dimensions: int = 1536, similarity: CosmosDBSimilarityType = CosmosDBSimilarityType.COS) → dict[str, Any][source]¶
Creates an index using the index name specified atinstance construction
Setting the numLists parameter correctly is important for achievinggood accuracy and performance.
Since the vector store uses IVF as the indexing strategy,
you should create the index only after you
have loaded a large enough sample documents to ensure that the
centroids for the respective buckets are
faily distributed.
We recommend that numLists is set to documentCount/1000 for upto 1 million documents
and to sqrt(documentCount) for more than 1 million documents.
As the number of items in your database grows, you should
tune numLists to be larger
in order to achieve good latency performance for vector search.
If you’re experimenting with a new scenario or creating a
small demo, you can start with numLists
set to 1 to perform a brute-force search across all vectors.
This should provide you with the most
accurate results from the vector search, however be aware that
the search speed and latency will be slow.


the search speed and latency will be slow.
After your initial setup, you should go ahead and tune
the numLists parameter using the above guidance.
Parameters
num_lists – This integer is the number of clusters that the
inverted file (IVF) index uses to group the vector data.
We recommend that numLists is set to documentCount/1000
for up to 1 million documents and to sqrt(documentCount)
for more than 1 million documents.
Using a numLists value of 1 is akin to performing
brute-force search, which has limited performance
dimensions – Number of dimensions for vector similarity.
The maximum number of supported dimensions is 2000
similarity – Similarity metric to use with the IVF index.
Possible options are:
CosmosDBSimilarityType.COS (cosine distance),
CosmosDBSimilarityType.L2 (Euclidean distance), and
CosmosDBSimilarityType.IP (inner product).
Returns
An object describing the created index
delete(ids: Optional[List[str]] = None, **kwargs: Any) → Optional[bool][source]¶
Delete by vector ID or other criteria.
Parameters
ids – List of ids to delete.
**kwargs – Other keyword arguments that subclasses might use.
Returns
True if deletion is successful,
False otherwise, None if not implemented.
Return type
Optional[bool]
delete_document_by_id(document_id: Optional[str] = None) → None[source]¶
Removes a Specific Document by Id
Parameters
document_id – The document identifier
delete_index() → None[source]¶
Deletes the index specified during instance construction if it exists
classmethod from_connection_string(connection_string: str, namespace: str, embedding: Embeddings, **kwargs: Any) → AzureCosmosDBVectorSearch[source]¶


Creates an Instance of AzureCosmosDBVectorSearch from a Connection String
Parameters
connection_string – The MongoDB vCore instance connection string
namespace – The namespace (database.collection)
embedding – The embedding utility
**kwargs – Dynamic keyword arguments
Returns
an instance of the vector store
classmethod from_documents(documents: List[Document], embedding: Embeddings, **kwargs: Any) → VST¶
Return VectorStore initialized from documents and embeddings.
classmethod from_texts(texts: List[str], embedding: Embeddings, metadatas: Optional[List[dict]] = None, collection: Optional[Collection[CosmosDBDocumentType]] = None, **kwargs: Any) → AzureCosmosDBVectorSearch[source]¶
Return VectorStore initialized from texts and embeddings.
get_index_name() → str[source]¶
Returns the index name
Returns
Returns the index name
index_exists() → bool[source]¶
Verifies if the specified index name during instanceconstruction exists on the collection
Returns
Returns True on success and False if no such index existson the collection
max_marginal_relevance_search(query: str, k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, **kwargs: Any) → List[Document][source]¶
Return docs selected using the maximal marginal relevance.
Maximal marginal relevance optimizes for similarity to query AND diversity
among selected documents.
Parameters
query – Text to look up documents similar to.
k – Number of Documents to return. Defaults to 4.
fetch_k – Number of Documents to fetch to pass to MMR algorithm.
lambda_mult – Number between 0 and 1 that determines the degree
of diversity among the results with 0 corresponding
to maximum diversity and 1 to minimum diversity.
Defaults to 0.5.
Returns


Defaults to 0.5.
Returns
List of Documents selected by maximal marginal relevance.
max_marginal_relevance_search_by_vector(embedding: List[float], k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, **kwargs: Any) → List[Document][source]¶
Return docs selected using the maximal marginal relevance.
Maximal marginal relevance optimizes for similarity to query AND diversity
among selected documents.
Parameters
embedding – Embedding to look up documents similar to.
k – Number of Documents to return. Defaults to 4.
fetch_k – Number of Documents to fetch to pass to MMR algorithm.
lambda_mult – Number between 0 and 1 that determines the degree
of diversity among the results with 0 corresponding
to maximum diversity and 1 to minimum diversity.
Defaults to 0.5.
Returns
List of Documents selected by maximal marginal relevance.
search(query: str, search_type: str, **kwargs: Any) → List[Document]¶
Return docs most similar to query using specified search type.
similarity_search(query: str, k: int = 4, **kwargs: Any) → List[Document][source]¶
Return docs most similar to query.
similarity_search_by_vector(embedding: List[float], k: int = 4, **kwargs: Any) → List[Document]¶
Return docs most similar to embedding vector.
Parameters
embedding – Embedding to look up documents similar to.
k – Number of Documents to return. Defaults to 4.
Returns
List of Documents most similar to the query vector.
similarity_search_with_relevance_scores(query: str, k: int = 4, **kwargs: Any) → List[Tuple[Document, float]]¶


Return docs and relevance scores in the range [0, 1].
0 is dissimilar, 1 is most similar.
Parameters
query – input text
k – Number of Documents to return. Defaults to 4.
**kwargs – kwargs to be passed to similarity search. Should include:
score_threshold: Optional, a floating point value between 0 to 1 to
filter the resulting set of retrieved docs
Returns
List of Tuples of (doc, similarity_score)
similarity_search_with_score(query: str, k: int = 4) → List[Tuple[Document, float]][source]¶
Run similarity search with distance.


langchain.vectorstores.clickhouse.Clickhouse¶
class langchain.vectorstores.clickhouse.Clickhouse(embedding: Embeddings, config: Optional[ClickhouseSettings] = None, **kwargs: Any)[source]¶
ClickHouse VectorSearch vector store.
You need a clickhouse-connect python package, and a valid account
to connect to ClickHouse.
ClickHouse can not only search with simple vector indexes,
it also supports complex query with multiple conditions,
constraints and even sub-queries.
For more information, please visit[ClickHouse official site](https://clickhouse.com/clickhouse)
ClickHouse Wrapper to LangChain
embedding_function (Embeddings):
config (ClickHouseSettings): Configuration to ClickHouse Client
Other keyword arguments will pass into
[clickhouse-connect](https://docs.clickhouse.com/)
Attributes
embeddings
Access the query embedding object if available.
metadata_column
Methods
__init__(embedding[, config])
ClickHouse Wrapper to LangChain
aadd_documents(documents, **kwargs)
Run more documents through the embeddings and add to the vectorstore.
aadd_texts(texts[, metadatas])
Run more texts through the embeddings and add to the vectorstore.
add_documents(documents, **kwargs)
Run more documents through the embeddings and add to the vectorstore.
add_texts(texts[, metadatas, batch_size, ids])
Insert more texts through the embeddings and add to the VectorStore.
adelete([ids])
Delete by vector ID or other criteria.
afrom_documents(documents, embedding, **kwargs)
Return VectorStore initialized from documents and embeddings.
afrom_texts(texts, embedding[, metadatas])
Return VectorStore initialized from texts and embeddings.
amax_marginal_relevance_search(query[, k, ...])


amax_marginal_relevance_search(query[, k, ...])
Return docs selected using the maximal marginal relevance.
amax_marginal_relevance_search_by_vector(...)
Return docs selected using the maximal marginal relevance.
as_retriever(**kwargs)
Return VectorStoreRetriever initialized from this VectorStore.
asearch(query, search_type, **kwargs)
Return docs most similar to query using specified search type.
asimilarity_search(query[, k])
Return docs most similar to query.
asimilarity_search_by_vector(embedding[, k])
Return docs most similar to embedding vector.
asimilarity_search_with_relevance_scores(query)
Return docs and relevance scores in the range [0, 1], asynchronously.
asimilarity_search_with_score(*args, **kwargs)
Run similarity search with distance asynchronously.
delete([ids])
Delete by vector ID or other criteria.
drop()
Helper function: Drop data
escape_str(value)
from_documents(documents, embedding, **kwargs)
Return VectorStore initialized from documents and embeddings.
from_texts(texts, embedding[, metadatas, ...])
Create ClickHouse wrapper with existing texts
max_marginal_relevance_search(query[, k, ...])
Return docs selected using the maximal marginal relevance.
max_marginal_relevance_search_by_vector(...)
Return docs selected using the maximal marginal relevance.
search(query, search_type, **kwargs)
Return docs most similar to query using specified search type.
similarity_search(query[, k, where_str])
Perform a similarity search with ClickHouse
similarity_search_by_vector(embedding[, k, ...])
Perform a similarity search with ClickHouse by vectors
similarity_search_with_relevance_scores(query)
Perform a similarity search with ClickHouse


similarity_search_with_relevance_scores(query)
Perform a similarity search with ClickHouse
similarity_search_with_score(*args, **kwargs)
Run similarity search with distance.
__init__(embedding: Embeddings, config: Optional[ClickhouseSettings] = None, **kwargs: Any) → None[source]¶
ClickHouse Wrapper to LangChain
embedding_function (Embeddings):
config (ClickHouseSettings): Configuration to ClickHouse Client
Other keyword arguments will pass into
[clickhouse-connect](https://docs.clickhouse.com/)
async aadd_documents(documents: List[Document], **kwargs: Any) → List[str]¶
Run more documents through the embeddings and add to the vectorstore.
Parameters
(List[Document] (documents) – Documents to add to the vectorstore.
Returns
List of IDs of the added texts.
Return type
List[str]
async aadd_texts(texts: Iterable[str], metadatas: Optional[List[dict]] = None, **kwargs: Any) → List[str]¶
Run more texts through the embeddings and add to the vectorstore.
add_documents(documents: List[Document], **kwargs: Any) → List[str]¶
Run more documents through the embeddings and add to the vectorstore.
Parameters
(List[Document] (documents) – Documents to add to the vectorstore.
Returns
List of IDs of the added texts.
Return type
List[str]
add_texts(texts: Iterable[str], metadatas: Optional[List[dict]] = None, batch_size: int = 32, ids: Optional[Iterable[str]] = None, **kwargs: Any) → List[str][source]¶
Insert more texts through the embeddings and add to the VectorStore.
Parameters
texts – Iterable of strings to add to the VectorStore.
ids – Optional list of ids to associate with the texts.


ids – Optional list of ids to associate with the texts.
batch_size – Batch size of insertion
metadata – Optional column data to be inserted
Returns
List of ids from adding the texts into the VectorStore.
async adelete(ids: Optional[List[str]] = None, **kwargs: Any) → Optional[bool]¶
Delete by vector ID or other criteria.
Parameters
ids – List of ids to delete.
**kwargs – Other keyword arguments that subclasses might use.
Returns
True if deletion is successful,
False otherwise, None if not implemented.
Return type
Optional[bool]
async classmethod afrom_documents(documents: List[Document], embedding: Embeddings, **kwargs: Any) → VST¶
Return VectorStore initialized from documents and embeddings.
async classmethod afrom_texts(texts: List[str], embedding: Embeddings, metadatas: Optional[List[dict]] = None, **kwargs: Any) → VST¶
Return VectorStore initialized from texts and embeddings.
async amax_marginal_relevance_search(query: str, k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, **kwargs: Any) → List[Document]¶
Return docs selected using the maximal marginal relevance.
async amax_marginal_relevance_search_by_vector(embedding: List[float], k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, **kwargs: Any) → List[Document]¶
Return docs selected using the maximal marginal relevance.
as_retriever(**kwargs: Any) → VectorStoreRetriever¶
Return VectorStoreRetriever initialized from this VectorStore.
Parameters
search_type (Optional[str]) – Defines the type of search that
the Retriever should perform.


the Retriever should perform.
Can be “similarity” (default), “mmr”, or
“similarity_score_threshold”.
search_kwargs (Optional[Dict]) – Keyword arguments to pass to the
search function. Can include things like:
k: Amount of documents to return (Default: 4)
score_threshold: Minimum relevance threshold
for similarity_score_threshold
fetch_k: Amount of documents to pass to MMR algorithm (Default: 20)
lambda_mult: Diversity of results returned by MMR;
1 for minimum diversity and 0 for maximum. (Default: 0.5)
filter: Filter by document metadata
Returns
Retriever class for VectorStore.
Return type
VectorStoreRetriever
Examples:
# Retrieve more documents with higher diversity
# Useful if your dataset has many similar documents
docsearch.as_retriever(
    search_type="mmr",
    search_kwargs={'k': 6, 'lambda_mult': 0.25}
)
# Fetch more documents for the MMR algorithm to consider
# But only return the top 5
docsearch.as_retriever(
    search_type="mmr",
    search_kwargs={'k': 5, 'fetch_k': 50}
)
# Only retrieve documents that have a relevance score
# Above a certain threshold
docsearch.as_retriever(
    search_type="similarity_score_threshold",
    search_kwargs={'score_threshold': 0.8}
)
# Only get the single most similar document from the dataset
docsearch.as_retriever(search_kwargs={'k': 1})
# Use a filter to only retrieve documents from a specific paper
docsearch.as_retriever(
    search_kwargs={'filter': {'paper_title':'GPT-4 Technical Report'}}
)


search_kwargs={'filter': {'paper_title':'GPT-4 Technical Report'}}
)
async asearch(query: str, search_type: str, **kwargs: Any) → List[Document]¶
Return docs most similar to query using specified search type.
async asimilarity_search(query: str, k: int = 4, **kwargs: Any) → List[Document]¶
Return docs most similar to query.
async asimilarity_search_by_vector(embedding: List[float], k: int = 4, **kwargs: Any) → List[Document]¶
Return docs most similar to embedding vector.
async asimilarity_search_with_relevance_scores(query: str, k: int = 4, **kwargs: Any) → List[Tuple[Document, float]]¶
Return docs and relevance scores in the range [0, 1], asynchronously.
0 is dissimilar, 1 is most similar.
Parameters
query – input text
k – Number of Documents to return. Defaults to 4.
**kwargs – kwargs to be passed to similarity search. Should include:
score_threshold: Optional, a floating point value between 0 to 1 to
filter the resulting set of retrieved docs
Returns
List of Tuples of (doc, similarity_score)
async asimilarity_search_with_score(*args: Any, **kwargs: Any) → List[Tuple[Document, float]]¶
Run similarity search with distance asynchronously.
delete(ids: Optional[List[str]] = None, **kwargs: Any) → Optional[bool]¶
Delete by vector ID or other criteria.
Parameters
ids – List of ids to delete.
**kwargs – Other keyword arguments that subclasses might use.
Returns
True if deletion is successful,
False otherwise, None if not implemented.
Return type
Optional[bool]
drop() → None[source]¶


Return type
Optional[bool]
drop() → None[source]¶
Helper function: Drop data
escape_str(value: str) → str[source]¶
classmethod from_documents(documents: List[Document], embedding: Embeddings, **kwargs: Any) → VST¶
Return VectorStore initialized from documents and embeddings.
classmethod from_texts(texts: List[str], embedding: Embeddings, metadatas: Optional[List[Dict[Any, Any]]] = None, config: Optional[ClickhouseSettings] = None, text_ids: Optional[Iterable[str]] = None, batch_size: int = 32, **kwargs: Any) → Clickhouse[source]¶
Create ClickHouse wrapper with existing texts
Parameters
embedding_function (Embeddings) – Function to extract text embedding
texts (Iterable[str]) – List or tuple of strings to be added
config (ClickHouseSettings, Optional) – ClickHouse configuration
text_ids (Optional[Iterable], optional) – IDs for the texts.
Defaults to None.
batch_size (int, optional) – Batchsize when transmitting data to ClickHouse.
Defaults to 32.
metadata (List[dict], optional) – metadata to texts. Defaults to None.
into (Other keyword arguments will pass) – [clickhouse-connect](https://clickhouse.com/docs/en/integrations/python#clickhouse-connect-driver-api)
Returns
ClickHouse Index
max_marginal_relevance_search(query: str, k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, **kwargs: Any) → List[Document]¶
Return docs selected using the maximal marginal relevance.
Maximal marginal relevance optimizes for similarity to query AND diversity
among selected documents.
Parameters
query – Text to look up documents similar to.
k – Number of Documents to return. Defaults to 4.


k – Number of Documents to return. Defaults to 4.
fetch_k – Number of Documents to fetch to pass to MMR algorithm.
lambda_mult – Number between 0 and 1 that determines the degree
of diversity among the results with 0 corresponding
to maximum diversity and 1 to minimum diversity.
Defaults to 0.5.
Returns
List of Documents selected by maximal marginal relevance.
max_marginal_relevance_search_by_vector(embedding: List[float], k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, **kwargs: Any) → List[Document]¶
Return docs selected using the maximal marginal relevance.
Maximal marginal relevance optimizes for similarity to query AND diversity
among selected documents.
Parameters
embedding – Embedding to look up documents similar to.
k – Number of Documents to return. Defaults to 4.
fetch_k – Number of Documents to fetch to pass to MMR algorithm.
lambda_mult – Number between 0 and 1 that determines the degree
of diversity among the results with 0 corresponding
to maximum diversity and 1 to minimum diversity.
Defaults to 0.5.
Returns
List of Documents selected by maximal marginal relevance.
search(query: str, search_type: str, **kwargs: Any) → List[Document]¶
Return docs most similar to query using specified search type.
similarity_search(query: str, k: int = 4, where_str: Optional[str] = None, **kwargs: Any) → List[Document][source]¶
Perform a similarity search with ClickHouse
Parameters
query (str) – query string
k (int, optional) – Top K neighbors to retrieve. Defaults to 4.
where_str (Optional[str], optional) – where condition string.
Defaults to None.


where_str (Optional[str], optional) – where condition string.
Defaults to None.
NOTE – Please do not let end-user to fill this and always be aware
of SQL injection. When dealing with metadatas, remember to
use {self.metadata_column}.attribute instead of attribute
alone. The default name for it is metadata.
Returns
List of Documents
Return type
List[Document]
similarity_search_by_vector(embedding: List[float], k: int = 4, where_str: Optional[str] = None, **kwargs: Any) → List[Document][source]¶
Perform a similarity search with ClickHouse by vectors
Parameters
query (str) – query string
k (int, optional) – Top K neighbors to retrieve. Defaults to 4.
where_str (Optional[str], optional) – where condition string.
Defaults to None.
NOTE – Please do not let end-user to fill this and always be aware
of SQL injection. When dealing with metadatas, remember to
use {self.metadata_column}.attribute instead of attribute
alone. The default name for it is metadata.
Returns
List of documents
Return type
List[Document]
similarity_search_with_relevance_scores(query: str, k: int = 4, where_str: Optional[str] = None, **kwargs: Any) → List[Tuple[Document, float]][source]¶
Perform a similarity search with ClickHouse
Parameters
query (str) – query string
k (int, optional) – Top K neighbors to retrieve. Defaults to 4.
where_str (Optional[str], optional) – where condition string.
Defaults to None.
NOTE – Please do not let end-user to fill this and always be aware
of SQL injection. When dealing with metadatas, remember to
use {self.metadata_column}.attribute instead of attribute


use {self.metadata_column}.attribute instead of attribute
alone. The default name for it is metadata.
Returns
List of (Document, similarity)
Return type
List[Document]
similarity_search_with_score(*args: Any, **kwargs: Any) → List[Tuple[Document, float]]¶
Run similarity search with distance.
Examples using Clickhouse¶
ClickHouse


langchain.vectorstores.deeplake.DeepLake¶
class langchain.vectorstores.deeplake.DeepLake(dataset_path: str = './deeplake/', token: Optional[str] = None, embedding: Optional[Embeddings] = None, embedding_function: Optional[Embeddings] = None, read_only: bool = False, ingestion_batch_size: int = 1000, num_workers: int = 0, verbose: bool = True, exec_option: Optional[str] = None, runtime: Optional[Dict] = None, index_params: Optional[Dict[str, Union[int, str]]] = None, **kwargs: Any)[source]¶
Activeloop Deep Lake vector store.
We integrated deeplake’s similarity search and filtering for fast prototyping.
Now, it supports Tensor Query Language (TQL) for production use cases
over billion rows.
Why Deep Lake?
Not only stores embeddings, but also the original data with version control.
Serverless, doesn’t require another service and can be used with majorcloud providers (S3, GCS, etc.)
More than just a multi-modal vector store. You can use the datasetto fine-tune your own LLM models.
To use, you should have the deeplake python package installed.
Example
from langchain.vectorstores import DeepLake
from langchain.embeddings.openai import OpenAIEmbeddings
embeddings = OpenAIEmbeddings()
vectorstore = DeepLake("langchain_store", embeddings.embed_query)
Creates an empty DeepLakeVectorStore or loads an existing one.
The DeepLakeVectorStore is located at the specified path.
Examples
>>> # Create a vector store with default tensors
>>> deeplake_vectorstore = DeepLake(
...        path = <path_for_storing_Data>,
... )
>>>
>>> # Create a vector store in the Deep Lake Managed Tensor Database


... )
>>>
>>> # Create a vector store in the Deep Lake Managed Tensor Database
>>> data = DeepLake(
...        path = "hub://org_id/dataset_name",
...        runtime = {"tensor_db": True},
... )
Parameters
dataset_path (str) – Path to existing dataset or where to create
a new one. Defaults to _LANGCHAIN_DEFAULT_DEEPLAKE_PATH.
token (str, optional) – Activeloop token, for fetching credentials
to the dataset at path if it is a Deep Lake dataset.
Tokens are normally autogenerated. Optional.
embedding (Embeddings, optional) – Function to convert
either documents or query. Optional.
embedding_function (Embeddings, optional) – Function to convert
either documents or query. Optional. Deprecated: keeping this
parameter for backwards compatibility.
read_only (bool) – Open dataset in read-only mode. Default is False.
ingestion_batch_size (int) – During data ingestion, data is divided
into batches. Batch size is the size of each batch.
Default is 1000.
num_workers (int) – Number of workers to use during data ingestion.
Default is 0.
verbose (bool) – Print dataset summary after each operation.
Default is True.
exec_option (str, optional) – DeepLakeVectorStore supports 3 ways to perform
searching - “python”, “compute_engine”, “tensor_db” and auto.
Default is None.
- auto- Selects the best execution method based on the storage
location of the Vector Store. It is the default option.
python - Pure-python implementation that runs on the client.WARNING: using this with big datasets can lead to memory
issues. Data can be stored anywhere.
compute_engine - C++ implementation of the Deep Lake ComputeEngine that runs on the client. Can be used for any data stored in


or connected to Deep Lake. Not for in-memory or local datasets.
tensor_db - Hosted Managed Tensor Database that isresponsible for storage and query execution. Only for data stored in
the Deep Lake Managed Database. Use runtime = {“db_engine”: True}
during dataset creation.
runtime (Dict, optional) – Parameters for creating the Vector Store in
Deep Lake’s Managed Tensor Database. Not applicable when loading an
existing Vector Store. To create a Vector Store in the Managed Tensor
Database, set runtime = {“tensor_db”: True}.
index_params (Optional[Dict[str, Union[int, str]]], optional) – Dictionary
containing information about vector index that will be created. Defaults
to None, which will utilize DEFAULT_VECTORSTORE_INDEX_PARAMS from
deeplake.constants. The specified key-values override the default
ones.
- threshold: The threshold for the dataset size above which an index
will be created for the embedding tensor. When the threshold value
is set to -1, index creation is turned off. Defaults to -1, which
turns off the index.
distance_metric: This key specifies the method of calculating thedistance between vectors when creating the vector database (VDB)
index. It can either be a string that corresponds to a member of
the DistanceType enumeration, or the string value itself.
- If no value is provided, it defaults to “L2”.
- “L2” corresponds to DistanceType.L2_NORM.
- “COS” corresponds to DistanceType.COSINE_SIMILARITY.
additional_params: Additional parameters for fine-tuning the index.
**kwargs – Other optional keyword arguments.
Raises
ValueError – If some condition is not met.
Attributes
embeddings
Access the query embedding object if available.
Methods
__init__([dataset_path, token, embedding, ...])


Methods
__init__([dataset_path, token, embedding, ...])
Creates an empty DeepLakeVectorStore or loads an existing one.
aadd_documents(documents, **kwargs)
Run more documents through the embeddings and add to the vectorstore.
aadd_texts(texts[, metadatas])
Run more texts through the embeddings and add to the vectorstore.
add_documents(documents, **kwargs)
Run more documents through the embeddings and add to the vectorstore.
add_texts(texts[, metadatas, ids])
Run more texts through the embeddings and add to the vectorstore.
adelete([ids])
Delete by vector ID or other criteria.
afrom_documents(documents, embedding, **kwargs)
Return VectorStore initialized from documents and embeddings.
afrom_texts(texts, embedding[, metadatas])
Return VectorStore initialized from texts and embeddings.
amax_marginal_relevance_search(query[, k, ...])
Return docs selected using the maximal marginal relevance.
amax_marginal_relevance_search_by_vector(...)
Return docs selected using the maximal marginal relevance.
as_retriever(**kwargs)
Return VectorStoreRetriever initialized from this VectorStore.
asearch(query, search_type, **kwargs)
Return docs most similar to query using specified search type.
asimilarity_search(query[, k])
Return docs most similar to query.
asimilarity_search_by_vector(embedding[, k])
Return docs most similar to embedding vector.
asimilarity_search_with_relevance_scores(query)
Return docs and relevance scores in the range [0, 1], asynchronously.
asimilarity_search_with_score(*args, **kwargs)
Run similarity search with distance asynchronously.
delete([ids])
Delete the entities in the dataset.
delete_dataset()


delete([ids])
Delete the entities in the dataset.
delete_dataset()
Delete the collection.
ds()
force_delete_by_path(path)
Force delete dataset by path.
from_documents(documents, embedding, **kwargs)
Return VectorStore initialized from documents and embeddings.
from_texts(texts[, embedding, metadatas, ...])
Create a Deep Lake dataset from a raw documents.
max_marginal_relevance_search(query[, k, ...])
Return docs selected using maximal marginal relevance.
max_marginal_relevance_search_by_vector(...)
Return docs selected using the maximal marginal relevance.
search(query, search_type, **kwargs)
Return docs most similar to query using specified search type.
similarity_search(query[, k])
Return docs most similar to query.
similarity_search_by_vector(embedding[, k])
Return docs most similar to embedding vector.
similarity_search_with_relevance_scores(query)
Return docs and relevance scores in the range [0, 1].
similarity_search_with_score(query[, k])
Run similarity search with Deep Lake with distance returned.
__init__(dataset_path: str = './deeplake/', token: Optional[str] = None, embedding: Optional[Embeddings] = None, embedding_function: Optional[Embeddings] = None, read_only: bool = False, ingestion_batch_size: int = 1000, num_workers: int = 0, verbose: bool = True, exec_option: Optional[str] = None, runtime: Optional[Dict] = None, index_params: Optional[Dict[str, Union[int, str]]] = None, **kwargs: Any) → None[source]¶
Creates an empty DeepLakeVectorStore or loads an existing one.
The DeepLakeVectorStore is located at the specified path.
Examples


The DeepLakeVectorStore is located at the specified path.
Examples
>>> # Create a vector store with default tensors
>>> deeplake_vectorstore = DeepLake(
...        path = <path_for_storing_Data>,
... )
>>>
>>> # Create a vector store in the Deep Lake Managed Tensor Database
>>> data = DeepLake(
...        path = "hub://org_id/dataset_name",
...        runtime = {"tensor_db": True},
... )
Parameters
dataset_path (str) – Path to existing dataset or where to create
a new one. Defaults to _LANGCHAIN_DEFAULT_DEEPLAKE_PATH.
token (str, optional) – Activeloop token, for fetching credentials
to the dataset at path if it is a Deep Lake dataset.
Tokens are normally autogenerated. Optional.
embedding (Embeddings, optional) – Function to convert
either documents or query. Optional.
embedding_function (Embeddings, optional) – Function to convert
either documents or query. Optional. Deprecated: keeping this
parameter for backwards compatibility.
read_only (bool) – Open dataset in read-only mode. Default is False.
ingestion_batch_size (int) – During data ingestion, data is divided
into batches. Batch size is the size of each batch.
Default is 1000.
num_workers (int) – Number of workers to use during data ingestion.
Default is 0.
verbose (bool) – Print dataset summary after each operation.
Default is True.
exec_option (str, optional) – DeepLakeVectorStore supports 3 ways to perform
searching - “python”, “compute_engine”, “tensor_db” and auto.
Default is None.
- auto- Selects the best execution method based on the storage
location of the Vector Store. It is the default option.


location of the Vector Store. It is the default option.
python - Pure-python implementation that runs on the client.WARNING: using this with big datasets can lead to memory
issues. Data can be stored anywhere.
compute_engine - C++ implementation of the Deep Lake ComputeEngine that runs on the client. Can be used for any data stored in
or connected to Deep Lake. Not for in-memory or local datasets.
tensor_db - Hosted Managed Tensor Database that isresponsible for storage and query execution. Only for data stored in
the Deep Lake Managed Database. Use runtime = {“db_engine”: True}
during dataset creation.
runtime (Dict, optional) – Parameters for creating the Vector Store in
Deep Lake’s Managed Tensor Database. Not applicable when loading an
existing Vector Store. To create a Vector Store in the Managed Tensor
Database, set runtime = {“tensor_db”: True}.
index_params (Optional[Dict[str, Union[int, str]]], optional) – Dictionary
containing information about vector index that will be created. Defaults
to None, which will utilize DEFAULT_VECTORSTORE_INDEX_PARAMS from
deeplake.constants. The specified key-values override the default
ones.
- threshold: The threshold for the dataset size above which an index
will be created for the embedding tensor. When the threshold value
is set to -1, index creation is turned off. Defaults to -1, which
turns off the index.
distance_metric: This key specifies the method of calculating thedistance between vectors when creating the vector database (VDB)
index. It can either be a string that corresponds to a member of
the DistanceType enumeration, or the string value itself.
- If no value is provided, it defaults to “L2”.
- “L2” corresponds to DistanceType.L2_NORM.
- “COS” corresponds to DistanceType.COSINE_SIMILARITY.


- “COS” corresponds to DistanceType.COSINE_SIMILARITY.
additional_params: Additional parameters for fine-tuning the index.
**kwargs – Other optional keyword arguments.
Raises
ValueError – If some condition is not met.
async aadd_documents(documents: List[Document], **kwargs: Any) → List[str]¶
Run more documents through the embeddings and add to the vectorstore.
Parameters
(List[Document] (documents) – Documents to add to the vectorstore.
Returns
List of IDs of the added texts.
Return type
List[str]
async aadd_texts(texts: Iterable[str], metadatas: Optional[List[dict]] = None, **kwargs: Any) → List[str]¶
Run more texts through the embeddings and add to the vectorstore.
add_documents(documents: List[Document], **kwargs: Any) → List[str]¶
Run more documents through the embeddings and add to the vectorstore.
Parameters
(List[Document] (documents) – Documents to add to the vectorstore.
Returns
List of IDs of the added texts.
Return type
List[str]
add_texts(texts: Iterable[str], metadatas: Optional[List[dict]] = None, ids: Optional[List[str]] = None, **kwargs: Any) → List[str][source]¶
Run more texts through the embeddings and add to the vectorstore.
Examples
>>> ids = deeplake_vectorstore.add_texts(
...     texts = <list_of_texts>,
...     metadatas = <list_of_metadata_jsons>,
...     ids = <list_of_ids>,
... )
Parameters
texts (Iterable[str]) – Texts to add to the vectorstore.
metadatas (Optional[List[dict]], optional) – Optional list of metadatas.


metadatas (Optional[List[dict]], optional) – Optional list of metadatas.
ids (Optional[List[str]], optional) – Optional list of IDs.
embedding_function (Optional[Embeddings], optional) – Embedding function
to use to convert the text into embeddings.
**kwargs (Any) – Any additional keyword arguments passed is not supported
by this method.
Returns
List of IDs of the added texts.
Return type
List[str]
async adelete(ids: Optional[List[str]] = None, **kwargs: Any) → Optional[bool]¶
Delete by vector ID or other criteria.
Parameters
ids – List of ids to delete.
**kwargs – Other keyword arguments that subclasses might use.
Returns
True if deletion is successful,
False otherwise, None if not implemented.
Return type
Optional[bool]
async classmethod afrom_documents(documents: List[Document], embedding: Embeddings, **kwargs: Any) → VST¶
Return VectorStore initialized from documents and embeddings.
async classmethod afrom_texts(texts: List[str], embedding: Embeddings, metadatas: Optional[List[dict]] = None, **kwargs: Any) → VST¶
Return VectorStore initialized from texts and embeddings.
async amax_marginal_relevance_search(query: str, k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, **kwargs: Any) → List[Document]¶
Return docs selected using the maximal marginal relevance.
async amax_marginal_relevance_search_by_vector(embedding: List[float], k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, **kwargs: Any) → List[Document]¶
Return docs selected using the maximal marginal relevance.


Return docs selected using the maximal marginal relevance.
as_retriever(**kwargs: Any) → VectorStoreRetriever¶
Return VectorStoreRetriever initialized from this VectorStore.
Parameters
search_type (Optional[str]) – Defines the type of search that
the Retriever should perform.
Can be “similarity” (default), “mmr”, or
“similarity_score_threshold”.
search_kwargs (Optional[Dict]) – Keyword arguments to pass to the
search function. Can include things like:
k: Amount of documents to return (Default: 4)
score_threshold: Minimum relevance threshold
for similarity_score_threshold
fetch_k: Amount of documents to pass to MMR algorithm (Default: 20)
lambda_mult: Diversity of results returned by MMR;
1 for minimum diversity and 0 for maximum. (Default: 0.5)
filter: Filter by document metadata
Returns
Retriever class for VectorStore.
Return type
VectorStoreRetriever
Examples:
# Retrieve more documents with higher diversity
# Useful if your dataset has many similar documents
docsearch.as_retriever(
    search_type="mmr",
    search_kwargs={'k': 6, 'lambda_mult': 0.25}
)
# Fetch more documents for the MMR algorithm to consider
# But only return the top 5
docsearch.as_retriever(
    search_type="mmr",
    search_kwargs={'k': 5, 'fetch_k': 50}
)
# Only retrieve documents that have a relevance score
# Above a certain threshold
docsearch.as_retriever(
    search_type="similarity_score_threshold",
    search_kwargs={'score_threshold': 0.8}
)
# Only get the single most similar document from the dataset


)
# Only get the single most similar document from the dataset
docsearch.as_retriever(search_kwargs={'k': 1})
# Use a filter to only retrieve documents from a specific paper
docsearch.as_retriever(
    search_kwargs={'filter': {'paper_title':'GPT-4 Technical Report'}}
)
async asearch(query: str, search_type: str, **kwargs: Any) → List[Document]¶
Return docs most similar to query using specified search type.
async asimilarity_search(query: str, k: int = 4, **kwargs: Any) → List[Document]¶
Return docs most similar to query.
async asimilarity_search_by_vector(embedding: List[float], k: int = 4, **kwargs: Any) → List[Document]¶
Return docs most similar to embedding vector.
async asimilarity_search_with_relevance_scores(query: str, k: int = 4, **kwargs: Any) → List[Tuple[Document, float]]¶
Return docs and relevance scores in the range [0, 1], asynchronously.
0 is dissimilar, 1 is most similar.
Parameters
query – input text
k – Number of Documents to return. Defaults to 4.
**kwargs – kwargs to be passed to similarity search. Should include:
score_threshold: Optional, a floating point value between 0 to 1 to
filter the resulting set of retrieved docs
Returns
List of Tuples of (doc, similarity_score)
async asimilarity_search_with_score(*args: Any, **kwargs: Any) → List[Tuple[Document, float]]¶
Run similarity search with distance asynchronously.
delete(ids: Optional[List[str]] = None, **kwargs: Any) → bool[source]¶
Delete the entities in the dataset.
Parameters


Delete the entities in the dataset.
Parameters
ids (Optional[List[str]], optional) – The document_ids to delete.
Defaults to None.
**kwargs – Other keyword arguments that subclasses might use.
- filter (Optional[Dict[str, str]], optional): The filter to delete by.
- delete_all (Optional[bool], optional): Whether to drop the dataset.
Returns
Whether the delete operation was successful.
Return type
bool
delete_dataset() → None[source]¶
Delete the collection.
ds() → Any[source]¶
classmethod force_delete_by_path(path: str) → None[source]¶
Force delete dataset by path.
Parameters
path (str) – path of the dataset to delete.
Raises
ValueError – if deeplake is not installed.
classmethod from_documents(documents: List[Document], embedding: Embeddings, **kwargs: Any) → VST¶
Return VectorStore initialized from documents and embeddings.
classmethod from_texts(texts: List[str], embedding: Optional[Embeddings] = None, metadatas: Optional[List[dict]] = None, ids: Optional[List[str]] = None, dataset_path: str = './deeplake/', **kwargs: Any) → DeepLake[source]¶
Create a Deep Lake dataset from a raw documents.
If a dataset_path is specified, the dataset will be persisted in that location,
otherwise by default at ./deeplake
Examples:
>>> # Search using an embedding
>>> vector_store = DeepLake.from_texts(
…        texts = <the_texts_that_you_want_to_embed>,
…        embedding_function = <embedding_function_for_query>,
…        k = <number_of_items_to_return>,
…        exec_option = <preferred_exec_option>,
… )
Parameters
dataset_path (str) –
The full path to the dataset. Can be:


dataset_path (str) –
The full path to the dataset. Can be:
Deep Lake cloud path of the form hub://username/dataset_name.To write to Deep Lake cloud datasets,
ensure that you are logged in to Deep Lake
(use ‘activeloop login’ from command line)
AWS S3 path of the form s3://bucketname/path/to/dataset.Credentials are required in either the environment
Google Cloud Storage path of the formgcs://bucketname/path/to/dataset Credentials are required
in either the environment
Local file system path of the form ./path/to/dataset or~/path/to/dataset or path/to/dataset.
In-memory path of the form mem://path/to/dataset which doesn’tsave the dataset, but keeps it in memory instead.
Should be used only for testing as it does not persist.
texts (List[Document]) – List of documents to add.
embedding (Optional[Embeddings]) – Embedding function. Defaults to None.
Note, in other places, it is called embedding_function.
metadatas (Optional[List[dict]]) – List of metadatas. Defaults to None.
ids (Optional[List[str]]) – List of document IDs. Defaults to None.
**kwargs – Additional keyword arguments.
Returns
Deep Lake dataset.
Return type
DeepLake
max_marginal_relevance_search(query: str, k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, exec_option: Optional[str] = None, **kwargs: Any) → List[Document][source]¶
Return docs selected using maximal marginal relevance.
Maximal marginal relevance optimizes for similarity to query AND diversity
among selected documents.
Examples:
>>> # Search using an embedding
>>> data = vector_store.max_marginal_relevance_search(


>>> data = vector_store.max_marginal_relevance_search(
…        query = <query_to_search>,
…        embedding_function = <embedding_function_for_query>,
…        k = <number_of_items_to_return>,
…        exec_option = <preferred_exec_option>,
… )
Parameters
query – Text to look up documents similar to.
k – Number of Documents to return. Defaults to 4.
fetch_k – Number of Documents for MMR algorithm.
lambda_mult – Value between 0 and 1. 0 corresponds
to maximum diversity and 1 to minimum.
Defaults to 0.5.
exec_option (str) – Supports 3 ways to perform searching.
- “python” - Pure-python implementation running on the client.
Can be used for data stored anywhere. WARNING: using this
option with big datasets is discouraged due to potential
memory issues.
”compute_engine” - Performant C++ implementation of the DeepLake Compute Engine. Runs on the client and can be used for
any data stored in or connected to Deep Lake. It cannot be
used with in-memory or local datasets.
”tensor_db” - Performant, fully-hosted Managed Tensor Database.Responsible for storage and query execution. Only available
for data stored in the Deep Lake Managed Database. To store
datasets in this database, specify
runtime = {“db_engine”: True} during dataset creation.
deep_memory (bool) – Whether to use the Deep Memory model for improving
search results. Defaults to False if deep_memory is not specified
in the Vector Store initialization. If True, the distance metric
is set to “deepmemory_distance”, which represents the metric with
which the model was trained. The search is performed using the Deep
Memory model. If False, the distance metric is set to “COS” or
whatever distance metric user specifies.


whatever distance metric user specifies.
**kwargs – Additional keyword arguments
Returns
List of Documents selected by maximal marginal relevance.
Raises
ValueError – when MRR search is on but embedding function is
    not specified.
max_marginal_relevance_search_by_vector(embedding: List[float], k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, exec_option: Optional[str] = None, **kwargs: Any) → List[Document][source]¶
Return docs selected using the maximal marginal relevance. Maximal marginal
relevance optimizes for similarity to query AND diversity among selected docs.
Examples:
>>> data = vector_store.max_marginal_relevance_search_by_vector(
…        embedding=<your_embedding>,
…        fetch_k=<elements_to_fetch_before_mmr_search>,
…        k=<number_of_items_to_return>,
…        exec_option=<preferred_exec_option>,
… )
Parameters
embedding – Embedding to look up documents similar to.
k – Number of Documents to return. Defaults to 4.
fetch_k – Number of Documents to fetch for MMR algorithm.
lambda_mult – Number between 0 and 1 determining the degree of diversity.
0 corresponds to max diversity and 1 to min diversity. Defaults to 0.5.
exec_option (str) – DeepLakeVectorStore supports 3 ways for searching.
Could be “python”, “compute_engine” or “tensor_db”. Defaults to
“python”.
- “python” - Pure-python implementation running on the client.
Can be used for data stored anywhere. WARNING: using this
option with big datasets is discouraged due to potential
memory issues.
”compute_engine” - Performant C++ implementation of the DeepLake Compute Engine. Runs on the client and can be used for


any data stored in or connected to Deep Lake. It cannot be used
with in-memory or local datasets.
”tensor_db” - Performant, fully-hosted Managed Tensor Database.Responsible for storage and query execution. Only available for
data stored in the Deep Lake Managed Database. To store datasets
in this database, specify runtime = {“db_engine”: True}
during dataset creation.
deep_memory (bool) – Whether to use the Deep Memory model for improving
search results. Defaults to False if deep_memory is not specified
in the Vector Store initialization. If True, the distance metric
is set to “deepmemory_distance”, which represents the metric with
which the model was trained. The search is performed using the Deep
Memory model. If False, the distance metric is set to “COS” or
whatever distance metric user specifies.
**kwargs – Additional keyword arguments.
Returns
List[Documents] - A list of documents.
search(query: str, search_type: str, **kwargs: Any) → List[Document]¶
Return docs most similar to query using specified search type.
similarity_search(query: str, k: int = 4, **kwargs: Any) → List[Document][source]¶
Return docs most similar to query.
Examples
>>> # Search using an embedding
>>> data = vector_store.similarity_search(
...     query=<your_query>,
...     k=<num_items>,
...     exec_option=<preferred_exec_option>,
... )
>>> # Run tql search:
>>> data = vector_store.similarity_search(
...     query=None,
...     tql="SELECT * WHERE id == <id>",
...     exec_option="compute_engine",
... )
Parameters
k (int) – Number of Documents to return. Defaults to 4.
query (str) – Text to look up similar documents.


query (str) – Text to look up similar documents.
**kwargs – Additional keyword arguments include:
embedding (Callable): Embedding function to use. Defaults to None.
distance_metric (str): ‘L2’ for Euclidean, ‘L1’ for Nuclear, ‘max’
for L-infinity, ‘cos’ for cosine, ‘dot’ for dot product.
Defaults to ‘L2’.
filter (Union[Dict, Callable], optional): Additional filterbefore embedding search.
- Dict: Key-value search on tensors of htype json,
(sample must satisfy all key-value filters)
Dict = {“tensor_1”: {“key”: value}, “tensor_2”: {“key”: value}}
Function: Compatible with deeplake.filter.
Defaults to None.
exec_option (str): Supports 3 ways to perform searching.’python’, ‘compute_engine’, or ‘tensor_db’. Defaults to ‘python’.
- ‘python’: Pure-python implementation for the client.
WARNING: not recommended for big datasets.
’compute_engine’: C++ implementation of the Compute Engine forthe client. Not for in-memory or local datasets.
’tensor_db’: Managed Tensor Database for storage and query.Only for data in Deep Lake Managed Database.
Use runtime = {“db_engine”: True} during dataset creation.
deep_memory (bool): Whether to use the Deep Memory model for improvingsearch results. Defaults to False if deep_memory is not specified
in the Vector Store initialization. If True, the distance metric
is set to “deepmemory_distance”, which represents the metric with
which the model was trained. The search is performed using the Deep
Memory model. If False, the distance metric is set to “COS” or
whatever distance metric user specifies.
Returns
List of Documents most similar to the query vector.
Return type
List[Document]


List of Documents most similar to the query vector.
Return type
List[Document]
similarity_search_by_vector(embedding: Union[List[float], ndarray], k: int = 4, **kwargs: Any) → List[Document][source]¶
Return docs most similar to embedding vector.
Examples
>>> # Search using an embedding
>>> data = vector_store.similarity_search_by_vector(
...    embedding=<your_embedding>,
...    k=<num_items_to_return>,
...    exec_option=<preferred_exec_option>,
... )
Parameters
embedding (Union[List[float], np.ndarray]) – Embedding to find similar docs.
k (int) – Number of Documents to return. Defaults to 4.
**kwargs – Additional keyword arguments including:
filter (Union[Dict, Callable], optional):
Additional filter before embedding search.
- Dict - Key-value search on tensors of htype json. True
if all key-value filters are satisfied.
Dict = {“tensor_name_1”: {“key”: value},
”tensor_name_2”: {“key”: value}}
Function - Any function compatible withdeeplake.filter.
Defaults to None.
exec_option (str): Options for search execution include”python”, “compute_engine”, or “tensor_db”. Defaults to
“python”.
- “python” - Pure-python implementation running on the client.
Can be used for data stored anywhere. WARNING: using this
option with big datasets is discouraged due to potential
memory issues.
”compute_engine” - Performant C++ implementation of the DeepLake Compute Engine. Runs on the client and can be used for
any data stored in or connected to Deep Lake. It cannot be
used with in-memory or local datasets.
”tensor_db” - Performant, fully-hosted Managed Tensor Database.Responsible for storage and query execution. Only available


for data stored in the Deep Lake Managed Database.
To store datasets in this database, specify
runtime = {“db_engine”: True} during dataset creation.
distance_metric (str): L2 for Euclidean, L1 for Nuclear,max for L-infinity distance, cos for cosine similarity,
‘dot’ for dot product. Defaults to L2.
deep_memory (bool): Whether to use the Deep Memory model for improvingsearch results. Defaults to False if deep_memory is not specified
in the Vector Store initialization. If True, the distance metric
is set to “deepmemory_distance”, which represents the metric with
which the model was trained. The search is performed using the Deep
Memory model. If False, the distance metric is set to “COS” or
whatever distance metric user specifies.
Returns
List of Documents most similar to the query vector.
Return type
List[Document]
similarity_search_with_relevance_scores(query: str, k: int = 4, **kwargs: Any) → List[Tuple[Document, float]]¶
Return docs and relevance scores in the range [0, 1].
0 is dissimilar, 1 is most similar.
Parameters
query – input text
k – Number of Documents to return. Defaults to 4.
**kwargs – kwargs to be passed to similarity search. Should include:
score_threshold: Optional, a floating point value between 0 to 1 to
filter the resulting set of retrieved docs
Returns
List of Tuples of (doc, similarity_score)
similarity_search_with_score(query: str, k: int = 4, **kwargs: Any) → List[Tuple[Document, float]][source]¶
Run similarity search with Deep Lake with distance returned.
Examples:
>>> data = vector_store.similarity_search_with_score(
…     query=<your_query>,


>>> data = vector_store.similarity_search_with_score(
…     query=<your_query>,
…     embedding=<your_embedding_function>
…     k=<number_of_items_to_return>,
…     exec_option=<preferred_exec_option>,
… )
Parameters
query (str) – Query text to search for.
k (int) – Number of results to return. Defaults to 4.
**kwargs – Additional keyword arguments. Some of these arguments are:
distance_metric: L2 for Euclidean, L1 for Nuclear, max L-infinity
distance, cos for cosine similarity, ‘dot’ for dot product.
Defaults to L2.
filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None.embedding_function (Callable): Embedding function to use. Defaults
to None.
exec_option (str): DeepLakeVectorStore supports 3 ways to performsearching. It could be either “python”, “compute_engine” or
“tensor_db”. Defaults to “python”.
- “python” - Pure-python implementation running on the client.
Can be used for data stored anywhere. WARNING: using this
option with big datasets is discouraged due to potential
memory issues.
”compute_engine” - Performant C++ implementation of the DeepLake Compute Engine. Runs on the client and can be used for
any data stored in or connected to Deep Lake. It cannot be used
with in-memory or local datasets.
”tensor_db” - Performant, fully-hosted Managed Tensor Database.Responsible for storage and query execution. Only available for
data stored in the Deep Lake Managed Database. To store datasets
in this database, specify runtime = {“db_engine”: True}
during dataset creation.
deep_memory (bool): Whether to use the Deep Memory model for improvingsearch results. Defaults to False if deep_memory is not specified


in the Vector Store initialization. If True, the distance metric
is set to “deepmemory_distance”, which represents the metric with
which the model was trained. The search is performed using the Deep
Memory model. If False, the distance metric is set to “COS” or
whatever distance metric user specifies.
Returns
List of documents most similar to the querytext with distance in float.
Return type
List[Tuple[Document, float]]
Examples using DeepLake¶
Activeloop Deep Lake
Analysis of Twitter the-algorithm source code with LangChain, GPT4 and Activeloop’s Deep Lake
Use LangChain, GPT and Activeloop’s Deep Lake to work with code base
QA using Activeloop’s DeepLake
Deep Lake


langchain.vectorstores.redis.filters.RedisFilterExpression¶
class langchain.vectorstores.redis.filters.RedisFilterExpression(_filter: Optional[str] = None, operator: Optional[RedisFilterOperator] = None, left: Optional[RedisFilterExpression] = None, right: Optional[RedisFilterExpression] = None)[source]¶
A logical expression of RedisFilterFields.
RedisFilterExpressions can be combined using the & and | operators to create
complex logical expressions that evaluate to the Redis Query language.
This presents an interface by which users can create complex queries
without having to know the Redis Query language.
Filter expressions are not initialized directly. Instead they are built
by combining RedisFilterFields using the & and | operators.
Examples
>>> from langchain.vectorstores.redis import RedisTag, RedisNum
>>> brand_is_nike = RedisTag("brand") == "nike"
>>> price_is_under_100 = RedisNum("price") < 100
>>> filter = brand_is_nike & price_is_under_100
>>> print(str(filter))
(@brand:{nike} @price:[-inf (100)])
Methods
__init__([_filter, operator, left, right])
__init__(_filter: Optional[str] = None, operator: Optional[RedisFilterOperator] = None, left: Optional[RedisFilterExpression] = None, right: Optional[RedisFilterExpression] = None)[source]¶


langchain.vectorstores.elasticsearch.ElasticsearchStore¶
class langchain.vectorstores.elasticsearch.ElasticsearchStore(index_name: str, *, embedding: ~typing.Optional[~langchain.schema.embeddings.Embeddings] = None, es_connection: ~typing.Optional[Elasticsearch] = None, es_url: ~typing.Optional[str] = None, es_cloud_id: ~typing.Optional[str] = None, es_user: ~typing.Optional[str] = None, es_api_key: ~typing.Optional[str] = None, es_password: ~typing.Optional[str] = None, vector_query_field: str = 'vector', query_field: str = 'text', distance_strategy: ~typing.Optional[~typing.Literal[<DistanceStrategy.COSINE: 'COSINE'>, <DistanceStrategy.DOT_PRODUCT: 'DOT_PRODUCT'>, <DistanceStrategy.EUCLIDEAN_DISTANCE: 'EUCLIDEAN_DISTANCE'>]] = None, strategy: ~langchain.vectorstores.elasticsearch.BaseRetrievalStrategy = <langchain.vectorstores.elasticsearch.ApproxRetrievalStrategy object>)[source]¶
Elasticsearch vector store.
Example
from langchain.vectorstores import ElasticsearchStore
from langchain.embeddings.openai import OpenAIEmbeddings
embeddings = OpenAIEmbeddings()
vectorstore = ElasticsearchStore(
    embedding=OpenAIEmbeddings(),
    index_name="langchain-demo",
    es_url="http://localhost:9200"
)
Parameters
index_name – Name of the Elasticsearch index to create.
es_url – URL of the Elasticsearch instance to connect to.
cloud_id – Cloud ID of the Elasticsearch instance to connect to.
es_user – Username to use when connecting to Elasticsearch.
es_password – Password to use when connecting to Elasticsearch.
es_api_key – API key to use when connecting to Elasticsearch.
es_connection – Optional pre-existing Elasticsearch connection.


es_connection – Optional pre-existing Elasticsearch connection.
vector_query_field – Optional. Name of the field to store
the embedding vectors in.
query_field – Optional. Name of the field to store the texts in.
strategy – Optional. Retrieval strategy to use when searching the index.
Defaults to ApproxRetrievalStrategy. Can be one of
ExactRetrievalStrategy, ApproxRetrievalStrategy,
or SparseRetrievalStrategy.
distance_strategy – Optional. Distance strategy to use when
searching the index.
Defaults to COSINE. Can be one of COSINE,
EUCLIDEAN_DISTANCE, or DOT_PRODUCT.
If you want to use a cloud hosted Elasticsearch instance, you can pass in the
cloud_id argument instead of the es_url argument.
Example
from langchain.vectorstores import ElasticsearchStore
from langchain.embeddings.openai import OpenAIEmbeddings
vectorstore = ElasticsearchStore(
    embedding=OpenAIEmbeddings(),
    index_name="langchain-demo",
    es_cloud_id="<cloud_id>"
    es_user="elastic",
    es_password="<password>"
)
You can also connect to an existing Elasticsearch instance by passing in a
pre-existing Elasticsearch connection via the es_connection argument.
Example
from langchain.vectorstores import ElasticsearchStore
from langchain.embeddings.openai import OpenAIEmbeddings
from elasticsearch import Elasticsearch
es_connection = Elasticsearch("http://localhost:9200")
vectorstore = ElasticsearchStore(
    embedding=OpenAIEmbeddings(),
    index_name="langchain-demo",
    es_connection=es_connection
)
ElasticsearchStore by default uses the ApproxRetrievalStrategy, which uses the
HNSW algorithm to perform approximate nearest neighbor search. This is the
fastest and most memory efficient algorithm.
If you want to use the Brute force / Exact strategy for searching vectors, you


If you want to use the Brute force / Exact strategy for searching vectors, you
can pass in the ExactRetrievalStrategy to the ElasticsearchStore constructor.
Example
from langchain.vectorstores import ElasticsearchStore
from langchain.embeddings.openai import OpenAIEmbeddings
vectorstore = ElasticsearchStore(
    embedding=OpenAIEmbeddings(),
    index_name="langchain-demo",
    es_url="http://localhost:9200",
    strategy=ElasticsearchStore.ExactRetrievalStrategy()
)
Both strategies require that you know the similarity metric you want to use
when creating the index. The default is cosine similarity, but you can also
use dot product or euclidean distance.
Example
from langchain.vectorstores import ElasticsearchStore
from langchain.embeddings.openai import OpenAIEmbeddings
from langchain.vectorstores.utils import DistanceStrategy
vectorstore = ElasticsearchStore(
    embedding=OpenAIEmbeddings(),
    index_name="langchain-demo",
    es_url="http://localhost:9200",
    distance_strategy="DOT_PRODUCT"
)
Attributes
embeddings
Access the query embedding object if available.
Methods
ApproxRetrievalStrategy([query_model_id, ...])
Used to perform approximate nearest neighbor search using the HNSW algorithm.
ExactRetrievalStrategy()
Used to perform brute force / exact nearest neighbor search via script_score.
SparseVectorRetrievalStrategy([model_id])
Used to perform sparse vector search via text_expansion.
__init__(index_name, *[, embedding, ...])
aadd_documents(documents, **kwargs)
Run more documents through the embeddings and add to the vectorstore.
aadd_texts(texts[, metadatas])
Run more texts through the embeddings and add to the vectorstore.
add_documents(documents, **kwargs)


add_documents(documents, **kwargs)
Run more documents through the embeddings and add to the vectorstore.
add_embeddings(text_embeddings[, metadatas, ...])
Add the given texts and embeddings to the vectorstore.
add_texts(texts[, metadatas, ids, ...])
Run more texts through the embeddings and add to the vectorstore.
adelete([ids])
Delete by vector ID or other criteria.
afrom_documents(documents, embedding, **kwargs)
Return VectorStore initialized from documents and embeddings.
afrom_texts(texts, embedding[, metadatas])
Return VectorStore initialized from texts and embeddings.
amax_marginal_relevance_search(query[, k, ...])
Return docs selected using the maximal marginal relevance.
amax_marginal_relevance_search_by_vector(...)
Return docs selected using the maximal marginal relevance.
as_retriever(**kwargs)
Return VectorStoreRetriever initialized from this VectorStore.
asearch(query, search_type, **kwargs)
Return docs most similar to query using specified search type.
asimilarity_search(query[, k])
Return docs most similar to query.
asimilarity_search_by_vector(embedding[, k])
Return docs most similar to embedding vector.
asimilarity_search_with_relevance_scores(query)
Return docs and relevance scores in the range [0, 1], asynchronously.
asimilarity_search_with_score(*args, **kwargs)
Run similarity search with distance asynchronously.
connect_to_elasticsearch(*[, es_url, ...])
delete([ids, refresh_indices])
Delete documents from the Elasticsearch index.
from_documents(documents[, embedding, ...])
Construct ElasticsearchStore wrapper from documents.
from_texts(texts[, embedding, metadatas, ...])


from_texts(texts[, embedding, metadatas, ...])
Construct ElasticsearchStore wrapper from raw documents.
get_user_agent()
max_marginal_relevance_search(query[, k, ...])
Return docs selected using the maximal marginal relevance.
max_marginal_relevance_search_by_vector(...)
Return docs selected using the maximal marginal relevance.
search(query, search_type, **kwargs)
Return docs most similar to query using specified search type.
similarity_search(query[, k, fetch_k, filter])
Return Elasticsearch documents most similar to query.
similarity_search_by_vector(embedding[, k])
Return docs most similar to embedding vector.
similarity_search_by_vector_with_relevance_scores(...)
Return Elasticsearch documents most similar to query, along with scores.
similarity_search_with_relevance_scores(query)
Return docs and relevance scores in the range [0, 1].
similarity_search_with_score(query[, k, filter])
Return Elasticsearch documents most similar to query, along with scores.
static ApproxRetrievalStrategy(query_model_id: Optional[str] = None, hybrid: Optional[bool] = False, rrf: Optional[Union[dict, bool]] = True) → ApproxRetrievalStrategy[source]¶
Used to perform approximate nearest neighbor search
using the HNSW algorithm.
At build index time, this strategy will create a
dense vector field in the index and store the
embedding vectors in the index.
At query time, the text will either be embedded using the
provided embedding function or the query_model_id
will be used to embed the text using the model
deployed to Elasticsearch.
if query_model_id is used, do not provide an embedding function.
Parameters
query_model_id – Optional. ID of the model to use to
embed the query text within the stack. Requires


embed the query text within the stack. Requires
embedding model to be deployed to Elasticsearch.
hybrid – Optional. If True, will perform a hybrid search
using both the knn query and a text query.
Defaults to False.
rrf – Optional. rrf is Reciprocal Rank Fusion.
When hybrid is True,
and rrf is True, then rrf: {}.
and rrf is False, then rrf is omitted.
and isinstance(rrf, dict) is True, then pass in the dict values.
rrf could be passed for adjusting ‘rank_constant’ and ‘window_size’.
static ExactRetrievalStrategy() → ExactRetrievalStrategy[source]¶
Used to perform brute force / exact
nearest neighbor search via script_score.
static SparseVectorRetrievalStrategy(model_id: Optional[str] = None) → SparseRetrievalStrategy[source]¶
Used to perform sparse vector search via text_expansion.
Used for when you want to use ELSER model to perform document search.
At build index time, this strategy will create a pipeline that
will embed the text using the ELSER model and store the
resulting tokens in the index.
At query time, the text will be embedded using the ELSER
model and the resulting tokens will be used to
perform a text_expansion query.
Parameters
model_id – Optional. Default is “.elser_model_1”.
ID of the model to use to embed the query text
within the stack. Requires embedding model to be
deployed to Elasticsearch.