Long-term Memory Persistence, Enrichment, and Search for Langchain Apps
Langchain Python and LangchainJS ship with ZepMemory and ZepRetriever classes. There are also ZepVectorStore classes available for both Python and typescript.
Managing Chat History Memory
Zep's ZepMemory class can be used to provide long-term memory for your Langchain chat apps or agents. Zep will store the entire historical message stream, automatically summarize messages, enrich them with token counts, timestamps, metadata and more.
You can also provide your bot or agent with access to relevant messages in long-term storage by using Zep's built-in vector search.
Building Retrieval Augmented Generation Apps (Q&A over Docs)
Zep's ZepVectorStore class can be used to store a collection of documents, metadata, and related embeddings. Retrieval Augmented Generation (RAG) apps can then use Zep's vector search to surface documents relevant to a prompt.
Zep will automatically embed the documents using low-latency local models, ensuring that your app is fast and responsive.
Installing Zep
A Zep server install is required. You will also need to have the Zep Python SDK or Zep typescript SDK installed in order to use the Langchain ZepMemory and ZepRetriever classes.
See the Zep Quick Start Guide for details.
Using Zep as a VectorStore and Document Retriever
See the examples in the inline Document Vector Store documentation and the LangChain.js documentation.
Using Zep as a LangChain Memory Store
Using Zep as your Langchain app's long-term memory simple: initialize the ZepMemory with your Zep instance URL, API key, and your user's session identifier.
ZepMemory instantiation
from langchain.memory import ZepMemory
# Set this to your Zep server URL
ZEP_API_URL = "http://localhost:8000"
ZEP_API_KEY = "<your JWT token>" # optional
session_id = str(uuid4()) # This is a unique identifier for the user
# Set up ZepMemory instance
memory = ZepMemory(
session_id=session_id,
url=ZEP_API_URL,
api_key=zep_api_key,
memory_key="chat_history",
)
Using Zep as your Langchain app's long-term memory simple: initialize the ZepMemory with your Zep instance URL and your user's session identifier (see Zep Concepts) and then utilize it as the chat memory for a Langchain Conversation.
ZepMemory instantiation
import { ChatOpenAI } from "langchain/chat_models/openai";
import { ConversationChain } from "langchain/chains";
import { ZepMemory } from "langchain/memory/zep";
import { v4 as uuidv4 } from 'uuid'
// Set this to your Zep server URL
const zepApiURL = "http://localhost:8000";
const zepApiKey = "<your JWT token>"; // optional
const sessionId = uuid4(); // This is a unique identifier for the user
// Set up Zep Memory
const memory = new ZepMemory({
sessionId,
baseURL: zepApiURL,
apiKey: zepApiKey,
});
Once you've created the memory, use it in your chain or with your agent.
Running a chain with ZepMemory
llm = ChatOpenAI(model_name="gpt-3.5-turbo")
chain = ConversationChain(llm=llm, verbose=True, memory=memory)
chain.run(
input="What is the book's relevance to the challenges facing contemporary society?"
)
Output:
> Entering new AgentExecutor chain...
AI: Parable of the Sower is a powerful exploration of the challenges facing contemporary society, such as environmental disasters, poverty, and violence. It examines how these issues can lead to the breakdown of society and how individuals can take action to create a better future. The novel also explores themes of faith, hope, and resilience in the face of adversity.
> Finished chain.
'Parable of the Sower is a powerful exploration of the challenges facing contemporary society, such as environmental disasters, poverty, and violence. It examines how these issues can lead to the breakdown of society and how individuals can take action to create a better future. The novel also explores themes of faith, hope, and resilience in the face of adversity.'
Inspecting the zep_chat_history, we see that a summary was automatically created, and that the history has been enriched with token counts, UUIDs, and timestamps.
Inspecting the Zep memory
def print_messages(messages):
for m in messages:
print(m.type, ":\n", m.dict())
print(memory.chat_memory.zep_summary)
print("\n")
print_messages(memory.chat_memory.messages)
The human inquires about Octavia Butler. The AI identifies her as an American science fiction author. The human then asks which books of hers were made into movies. The AI responds by mentioning the FX series Kindred, based on her novel of the same name. The human then asks about her contemporaries, and the AI lists Ursula K. Le Guin, Samuel R. Delany, and Joanna Russ.
system :
{'content': 'The human inquires about Octavia Butler. The AI identifies her as an American science fiction author. The human then asks which books of hers were made into movies. The AI responds by mentioning the FX series Kindred, based on her novel of the same name. The human then asks about her contemporaries, and the AI lists Ursula K. Le Guin, Samuel R. Delany, and Joanna Russ.', 'additional_kwargs': {}}
human :
{'content': 'What awards did she win?', 'additional_kwargs': {'uuid': '6b733f0b-6778-49ae-b3ec-4e077c039f31', 'created_at': '2023-07-09T19:23:16.611232Z', 'token_count': 8, 'metadata': {'system': {'entities': [], 'intent': 'The subject is inquiring about the awards that someone, whose identity is not specified, has won.'}}}, 'example': False}
ai :
{'content': 'Octavia Butler won the Hugo Award, the Nebula Award, and the MacArthur Fellowship.', 'additional_kwargs': {'uuid': '2f6d80c6-3c08-4fd4-8d4e-7bbee341ac90', 'created_at': '2023-07-09T19:23:16.618947Z', 'token_count': 21, 'metadata': {'system': {'entities': [{'Label': 'PERSON', 'Matches': [{'End': 14, 'Start': 0, 'Text': 'Octavia Butler'}], 'Name': 'Octavia Butler'}, {'Label': 'WORK_OF_ART', 'Matches': [{'End': 33, 'Start': 19, 'Text': 'the Hugo Award'}], 'Name': 'the Hugo Award'}, {'Label': 'EVENT', 'Matches': [{'End': 81, 'Start': 57, 'Text': 'the MacArthur Fellowship'}], 'Name': 'the MacArthur Fellowship'}], 'intent': 'The subject is stating that Octavia Butler received the Hugo Award, the Nebula Award, and the MacArthur Fellowship.'}}}, 'example': False}
human :
{'content': 'Which other women sci-fi writers might I want to read?', 'additional_kwargs': {'uuid': 'ccdcc901-ea39-4981-862f-6fe22ab9289b', 'created_at': '2023-07-09T19:23:16.62678Z', 'token_count': 14, 'metadata': {'system': {'entities': [], 'intent': 'The subject is seeking recommendations for additional women science fiction writers to explore.'}}}, 'example': False}
<snip />
Running a chain with ZepMemory
const chain = new ConversationChain({ llm: model, memory });
const response = await chain.run(
{
input="What is the book's relevance to the challenges facing contemporary society?"
},
)
Output:
> Entering new AgentExecutor chain...
AI: Parable of the Sower is a powerful exploration of the challenges facing contemporary society, such as environmental disasters, poverty, and violence. It examines how these issues can lead to the breakdown of society and how individuals can take action to create a better future. The novel also explores themes of faith, hope, and resilience in the face of adversity.
> Finished chain.
'Parable of the Sower is a powerful exploration of the challenges facing contemporary society, such as environmental disasters, poverty, and violence. It examines how these issues can lead to the breakdown of society and how individuals can take action to create a better future. The novel also explores themes of faith, hope, and resilience in the face of adversity.'
Inspecting the ZepMemory, we see that a summary was automatically created, and that the history has been enriched with token counts, UUIDs, and timestamps.
Inspecting the Zep Memory
const memoryVariables = await zepMemory.loadMemoryVariables({});
console.log(memoryVariables);
Output:
The AI provides a synopsis of Parable of the Sower, a science fiction novel by Octavia Butler, about a young woman navigating a dystopian future. The human asks for recommendations for other women sci-fi writers and the AI suggests Ursula K. Le Guin and Joanna Russ. The AI also notes that Butler was a highly acclaimed writer, having won the Hugo Award, the Nebula Award, and the MacArthur Fellowship.
{'uuid': 'becfe5f4-d24c-4487-b572-2e836d7cedc8', 'created_at': '2023-05-10T23:28:10.380343Z', 'role': 'human', 'content': "WWhat is the book's relevance to the challenges facing contemporary society?", 'token_count': 16}
{'uuid': '8ea4875c-bf42-4092-b3b8-308394963cb7', 'created_at': '2023-05-10T23:28:10.396994Z', 'role': 'ai', 'content': 'Parable of the Sower is a powerful exploration of the challenges facing contemporary society, such as environmental disasters, poverty, and violence. It examines how these issues can lead to the breakdown of society and how individuals can take action to create a better future. The novel also explores themes of faith, hope, and resilience in the face of adversity.', 'token_count': 0}
...
{'uuid': '2d95ff94-b52d-49bd-ade4-5e1e553e8cac', 'created_at': '2023-05-10T23:28:02.704311Z', 'role': 'ai', 'content': 'Octavia Estelle Butler (June 22, 1947 – February 24, 2006) was an American science fiction author.', 'token_count': 31}
Search Zep's message history from your Langchain app
Zep supports both vector search using a Langchain Retriever as well as via an instance of ZepMemory.
Using a ZepMemory instance
If you don't need to provide a Retriever to your chain or agent, you can search the long-term message history for a session directly from an instance of ZepMemory.
Search for relevant historical messages
search_results = memory.chat_memory.search("who are some famous women sci-fi authors?")
for r in search_results:
if r.dist > 0.8: # Only print results with similarity of 0.8 or higher
print(r.message, r.dist)
Output:
{'uuid': 'ccdcc901-ea39-4981-862f-6fe22ab9289b', 'created_at': '2023-07-09T19:23:16.62678Z', 'role': 'human', 'content': 'Which other women sci-fi writers might I want to read?', 'metadata': {'system': {'entities': [], 'intent': 'The subject is seeking recommendations for additional women science fiction writers to explore.'}}, 'token_count': 14} 0.9119619869747062
{'uuid': '7977099a-0c62-4c98-bfff-465bbab6c9c3', 'created_at': '2023-07-09T19:23:16.631721Z', 'role': 'ai', 'content': 'You might want to read Ursula K. Le Guin or Joanna Russ.', 'metadata': {'system': {'entities': [{'Label': 'ORG', 'Matches': [{'End': 40, 'Start': 23, 'Text': 'Ursula K. Le Guin'}], 'Name': 'Ursula K. Le Guin'}, {'Label': 'PERSON', 'Matches': [{'End': 55, 'Start': 44, 'Text': 'Joanna Russ'}], 'Name': 'Joanna Russ'}], 'intent': 'The subject is suggesting that the person should consider reading the works of Ursula K. Le Guin or Joanna Russ.'}}, 'token_count': 18} 0.8534346954749745
{'uuid': 'b05e2eb5-c103-4973-9458-928726f08655', 'created_at': '2023-07-09T19:23:16.603098Z', 'role': 'ai', 'content': "Octavia Butler's contemporaries included Ursula K. Le Guin, Samuel R. Delany, and Joanna Russ.", 'metadata': {'system': {'entities': [{'Label': 'PERSON', 'Matches': [{'End': 16, 'Start': 0, 'Text': "Octavia Butler's"}], 'Name': "Octavia Butler's"}, {'Label': 'ORG', 'Matches': [{'End': 58, 'Start': 41, 'Text': 'Ursula K. Le Guin'}], 'Name': 'Ursula K. Le Guin'}, {'Label': 'PERSON', 'Matches': [{'End': 76, 'Start': 60, 'Text': 'Samuel R. Delany'}], 'Name': 'Samuel R. Delany'}, {'Label': 'PERSON', 'Matches': [{'End': 93, 'Start': 82, 'Text': 'Joanna Russ'}], 'Name': 'Joanna Russ'}], 'intent': "The subject is stating that Octavia Butler's contemporaries included Ursula K. Le Guin, Samuel R. Delany, and Joanna Russ."}}, 'token_count': 27} 0.8523831524040919
Zep uses cosine distance for search ranking and the distance values are returned in the search results.
Using the ZepRetriever class
Retrievers are a powerful tool for in-context learning. That is, providing LLM models with context from external sources. Long-term chat message history is a valuable source of context for bots or agents that interact with humans over multiple engagements, potentially over months or years.
You wouldn't ordinarily call the Retriever directly, but we've done so as a simple illustration as to how Langchain documents returned by the ZepRetriever are enriched with metadata from the Zep service. In particular, token countx are useful when constructing prompts.
Use the ZepRetriever to search for relevant historical messages
from langchain.retrievers import ZepRetriever
zep_retriever = ZepRetriever(
# Ensure that you provide the session_id when instantiating the Retriever
session_id=session_id,
url=ZEP_API_URL,
top_k=5,
)
await zep_retriever.aget_relevant_documents("Who wrote Parable of the Sower?")
Output:
[Document(page_content='Parable of the Sower is a science fiction novel by Octavia Butler, published in 1993. It follows the story of Lauren Olamina, a young woman living in a dystopian future where society has collapsed due to environmental disasters, poverty, and violence.', metadata={'source': 'd2d364f3-d2cc-46be-9e04-b41333f80514', 'score': 0.8897276230611862, 'role': 'ai', 'token_count': 25, 'created_at': '2023-05-11T16:29:35.086398Z'}),
Document(page_content="Write a short synopsis of Butler's book, Parable of the Sower. What is it about?", metadata={'source': '2baee3f7-cfc8-4cce-9e9f-6e448249244b', 'score': 0.8858500630231024, 'role': 'human', 'token_count': 20, 'created_at': '2023-05-11T16:29:35.083828Z'}),
Document(page_content='Who was Octavia Butler?', metadata={'source': '73fb8cb3-da77-4f99-ac2b-939d5019e5b8', 'score': 0.7759728831215438, 'role': 'human', 'token_count': 0, 'created_at': '2023-05-11T16:29:35.014421Z'}),
Document(page_content="Octavia Butler's contemporaries included Ursula K. Le Guin, Samuel R. Delany, and Joanna Russ.", metadata={'source': 'aff1b45d-1e14-427d-a5a2-2b5a9dade294', 'score': 0.760286350496536, 'role': 'ai', 'token_count': 17, 'created_at': '2023-05-11T16:29:35.052896Z'}),
Document(page_content='You might want to read Ursula K. Le Guin or Joanna Russ.', metadata={'source': '0dd8cde5-860e-4d8b-975f-50f55028177d', 'score': 0.7595191167162665, 'role': 'ai', 'token_count': 15, 'created_at': '2023-05-11T16:29:35.080817Z'})]
Use the ZepRetriever to search for relevant historical messages
import { ZepRetriever } from "langchain/retrievers/zep";
export const run = async () => {
// Ensure that you provide the sessionId when instantiating the Retriever
const retriever = new ZepRetriever({ sessionId, ZEP_API_URL });
const query = "Who wrote Parable of the Sower?");
const docs = await retriever.getRelevantDocuments(query);
console.log(docs);
};
Output:
[Document(page_content='Parable of the Sower is a science fiction novel by Octavia Butler, published in 1993. It follows the story of Lauren Olamina, a young woman living in a dystopian future where society has collapsed due to environmental disasters, poverty, and violence.', metadata={'source': 'd2d364f3-d2cc-46be-9e04-b41333f80514', 'score': 0.8897276230611862, 'role': 'ai', 'token_count': 25, 'created_at': '2023-05-11T16:29:35.086398Z'}),
Document(page_content="Write a short synopsis of Butler's book, Parable of the Sower. What is it about?", metadata={'source': '2baee3f7-cfc8-4cce-9e9f-6e448249244b', 'score': 0.8858500630231024, 'role': 'human', 'token_count': 20, 'created_at': '2023-05-11T16:29:35.083828Z'}),
Document(page_content='Who was Octavia Butler?', metadata={'source': '73fb8cb3-da77-4f99-ac2b-939d5019e5b8', 'score': 0.7759728831215438, 'role': 'human', 'token_count': 0, 'created_at': '2023-05-11T16:29:35.014421Z'}),
Document(page_content="Octavia Butler's contemporaries included Ursula K. Le Guin, Samuel R. Delany, and Joanna Russ.", metadata={'source': 'aff1b45d-1e14-427d-a5a2-2b5a9dade294', 'score': 0.760286350496536, 'role': 'ai', 'token_count': 17, 'created_at': '2023-05-11T16:29:35.052896Z'}),
Document(page_content='You might want to read Ursula K. Le Guin or Joanna Russ.', metadata={'source': '0dd8cde5-860e-4d8b-975f-50f55028177d', 'score': 0.7595191167162665, 'role': 'ai', 'token_count': 15, 'created_at': '2023-05-11T16:29:35.080817Z'})]
````