You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
llm/05-Embedding/03-HuggingFace下载BGE模型实现向量转换.py

24 lines
842 B

This file contains ambiguous Unicode characters!

This file contains ambiguous Unicode characters that may be confused with others in your current locale. If your use case is intentional and legitimate, you can safely ignore this warning. Use the Escape button to highlight these characters.

# 使用这个类需要先安装库pip install langchain-huggingface
from langchain_huggingface import HuggingFaceEmbeddings
# 指定模型名,如果你本地没有这个模型,第一次执行后它会先下载
model_name = "BAAI/bge-small-zh-v1.5" # 模型名
model_kwargs = {'device': 'cpu'} # 没有显卡就用cpu有英伟达显卡写cuda
encode_kwargs = {'normalize_embeddings': True} # set True to compute cosine similarity
# 第一次运行会自动下载模型去huggingface上下载下载到hf默认的缓存目录。
hf_embedding = HuggingFaceEmbeddings(
model_name=model_name,
model_kwargs=model_kwargs,
encode_kwargs=encode_kwargs
)
resp = hf_embedding.embed_documents(
['I like large language models.',
'今天的天气非常不错!'
]
)
print(resp[0])
print(len(resp[0]))