我也遇到了向量化写本地时报KeyError: '__pydantic_fields_set__'的问题，

Question

模型用的是text-embedding-ada-002，这个问题依然存在，请问老师有什么思路吗？有没有可能是软件包的版本问题？
代码如下：
@app.post("/add_urls")
def add_urls(URL:str):
loader = WebBaseLoader(URL)
docs = loader.load()
docments = RecursiveCharacterTextSplitter(
chunk_size=800,
chunk_overlap=50,
).split_documents(docs)
#引入向量数据库
qdrant = Qdrant.from_documents(
docments,
OpenAIEmbeddings(model=“text-embedding-ada-002”),
path="./local_qdrand",
collection_name=“yunshi_2024”,
)
print(“向量数据库创建完成”)
return {“ok”: “添加成功！”}

报错如下（截取了有用信息）：
File “server.py”, line 218, in add_urls
qdrant = Qdrant.from_documents(
File “/home/ubuntu/.local/lib/python3.8/site-packages/langchain_core/vectorstores.py”, line 635, in from_documents
return cls.from_texts(texts, embedding, metadatas=metadatas, **kwargs)
File “/home/ubuntu/.local/lib/python3.8/site-packages/langchain_community/vectorstores/qdrant.py”, line 1339, in from_texts
qdrant = cls.construct_instance(
File “/home/ubuntu/.local/lib/python3.8/site-packages/langchain_community/vectorstores/qdrant.py”, line 1645, in construct_instance
client, async_client = cls._generate_clients(
File “/home/ubuntu/.local/lib/python3.8/site-packages/langchain_community/vectorstores/qdrant.py”, line 2250, in _generate_clients
sync_client = QdrantClient(
File “/home/ubuntu/.local/lib/python3.8/site-packages/qdrant_client/qdrant_client.py”, line 117, in init
self._client = QdrantLocal(
File “/home/ubuntu/.local/lib/python3.8/site-packages/qdrant_client/local/qdrant_local.py”, line 60, in init
self._load()
File “/home/ubuntu/.local/lib/python3.8/site-packages/qdrant_client/local/qdrant_local.py”, line 100, in _load
self.collections[collection_name] = LocalCollection(
File “/home/ubuntu/.local/lib/python3.8/site-packages/qdrant_client/local/local_collection.py”, line 130, in init
self.load_vectors()
File “/home/ubuntu/.local/lib/python3.8/site-packages/qdrant_client/local/local_collection.py”, line 142, in load_vectors
for idx, point in enumerate(self.storage.load()):
File “/home/ubuntu/.local/lib/python3.8/site-packages/qdrant_client/local/persistence.py”, line 152, in load
yield pickle.loads(row[0])
File “/home/ubuntu/.local/lib/python3.8/site-packages/pydantic/main.py”, line 816, in setstate
_object_setattr(self, ‘pydantic_fields_set’, state[‘pydantic_fields_set’])
KeyError: ‘pydantic_fields_set’

Answer 1

tomiezhang 2024-06-23 14:00:52

你的langchain和qdrant的版本号发一下，看样子像是pydantic的验证问题，如果你的版本是0.2以上最好回滚到0.1的大版本

• 提问者 weixin_慕工程5351025 #1

  langchain相关包版本信息如下：
  langchain                0.2.5               
  langchain-community      0.2.5               
  langchain-core           0.2.9               
  langchain-openai         0.0.5               
  langchain-text-splitters 0.2.1 
  
  
  
  qdrant版本信息如下：
  Name: qdrant-client
  Version: 1.9.2
  Summary: Client library for the Qdrant vector search engine
  Home-page: https://github.com/qdrant/qdrant-client
  Author: Andrey Vasnetsov
  Author-email: [e-mail]
  License: Apache-2.0
  Location: /home/ubuntu/.local/lib/python3.8/site-packages
  Requires: httpx, pydantic, grpcio-tools, numpy, portalocker, grpcio, urllib3
  Required-by:
  
  请问需要回滚的是哪个包的版本

• tomiezhang 回复 提问者 weixin_慕工程5351025 #2

  哦 那是了，你需要将langchain相关的包都降级到0.1大版本去，因为从0.2开始，官方摒弃了对pydantic 1.0的支持。
  “We’re planning a breaking release (0.3.0) sometime between July-September to upgrade to full support of Pydantic 2, and will drop support for Pydantic 1 (including objects originating from the v1 namespace of Pydantic 2).”
  官方的原话。