Okay folks, another one of my temporary side trips. My coding of a recurrent neural network is going okay. But it looks like it will need some 6 hours of training at a minimum. I really need to sort out how I want to go about that. And, what code changes may be required to facilitate my decision(s). So…

I do get asked if I have played around with ChatGPT and the like. I, in fact, haven’t really done so. One quick look quite some time back. So, I have decided to have a deeper look. But, I didn’t just want to play around without purpose. So, I have decided to look at coding a chatbot.

I will be using LangChain and LangGraph to help build the chatbot.

LangChain is a framework for developing applications powered by large language models (LLMs).
Introduction

I am going to start with MistralAI since it is completely free, no charge card required, and definitely good enough for an initial project. If I find it is too slow responding or it’s rate limits too restrictive, I may decide to look at paying for one of OpenAI’s models. But, that is not likely to happen in these early days.

Setup Account

May eventually add pictures and such but for now a really simple bit of an explanation.

  • go https://mistral.ai/
  • select the appropriate link to Build on la PlateForme (the page has changed in the weeks since I wrote the draft of this post, good luck)
  • sign-up (confirmations: e-mail, phone number)
  • select the free model
  • get an api key (don’t leave the popup modal without saving the key somewhere useful, e.g. .env file in the project directory; and make sure you put the .env file in your .gitignore if you are using git)

Setup Project Environment

Again, simple explanation (may add more detail later). But we’ve done this a few times now.

  • new directory: \learn\ds_llm
  • create a new conda environment: (base) PS R:\learn\ds_llm> conda create -n llm-3.13 python=3.13
  • activate the new environment
  • add project dependencies: (llm-3.13) PS R:\learn\ds_llm>conda install langchain langchain-mistralai langgraph python-dotenv -c conda-forge
  • add api key to .env file in project directory

Create Git Repository

(llm-3.13) PS R:\learn\ds_llm> git init
Initialized empty Git repository in R:/learn/ds_llm/.git/
(llm-3.13) PS R:\learn\ds_llm> git st
On branch master

No commits yet

Untracked files:
  (use "git add <file>..." to include in what will be committed)
        .env

I, a day or two later, created a private repository on GitHub and connected the local respository to it. I first did a local add and commit.

PS R:\learn\ds_llm> git remote add origin git@github.com:tooOld2Code/XXX.git
PS R:\learn\ds_llm> git branch -M main
PS R:\learn\ds_llm> git push -u origin main

Quick Test

Okay, let’s make sure we have everything working. A wee test script.

On the Langchain page referenced above, there is some sample code for a quick test. There is a drop down allowing one to select the model they wish to use. For MistralAI it looked like the following.

import getpass
import os

if not os.environ.get("MISTRAL_API_KEY"):
  os.environ["MISTRAL_API_KEY"] = getpass.getpass("Enter API key for Mistral AI: ")

from langchain_mistralai import ChatMistralAI

model = ChatMistralAI(model="mistral-large-latest")

model.invoke("Hello, world!")

There is at least one thing I will definitely change. You may have noticed python-dotenv amongst the libraries installed via conda above. I really don’t want to be manually passing the api key at the command line every time I run the file during development/testing.

import os
from dotenv import load_dotenv

from langchain_core.messages import HumanMessage
from langchain_mistralai.chat_models import ChatMistralAI

load_dotenv()

api_ky = os.getenv("MISTRAL_API_KEY")
model = ChatMistralAI(model="mistral-large-latest", api_key=api_ky)

llm_out = model.invoke("What is Python?")
print(llm_out)

And…

(llm-3.13) PS R:\learn\ds_llm> python api_tst.py
content="Python is a high-level, interpreted programming language that is widely used for a variety of applications. It was created by Guido van Rossum and first released in 1991. Python is known for its readability, simplicity, and versatility, which make it a popular choice for both beginners and experienced developers. Here are some key features and uses of Python:\n\n### Key Features:\n1. **Easy to Learn and Use**: Python's syntax is clean and easy to understand, which makes it a great language for beginners.\n2. **Interpreted Language**: Python code is executed line by line, which simplifies debugging and testing.\n3. **Dynamically Typed**: Variables do not need explicit declaration to reserve memory space. The declaration happens automatically when a value is assigned to a variable.\n4. **Extensive Standard Library**: Python comes with a large standard library that supports many common programming tasks, such as connecting to web servers, reading and modifying files, and more.\n5. **Cross-Platform**: Python is available on a wide variety of platforms including Windows, macOS, and various versions of Unix.\n6. **Object-Oriented**: Python supports object-oriented programming, which allows for better code organization and reuse.\n\n### Uses of Python:\n1. **Web Development**: Python is widely used for server-side web development. Frameworks like Django and Flask make web development faster and easier.\n2. **Data Science and Machine Learning**: Libraries like NumPy, Pandas, and SciPy, along with frameworks like TensorFlow and PyTorch, make Python a powerful tool for data analysis and machine learning.\n3. **Automation and Scripting**: Python is often used for automating tasks, such as web scraping, file manipulation, and process automation.\n4. **Software Development**: Python is used in the development of software applications, including desktop applications, web applications, and mobile applications.\n5. **Scientific Computing**: Python is used in scientific research for tasks like data analysis, visualization, and numerical computations.\n6. **Game Development**: Libraries like Pygame make it possible to develop games using Python.\n7. **Artificial Intelligence**: Python is widely used in AI for tasks like natural language processing, computer vision, and robotics.\n\n### Popular Python Libraries and Frameworks:\n- **NumPy**: For numerical computations.\n- **Pandas**: For data manipulation and analysis.\n- **Matplotlib**: For data visualization.\n- **Scikit-learn**: For machine learning.\n- **TensorFlow and PyTorch**: For deep learning.\n- **Django and Flask**: For web development.\n- **Pygame**: For game development.\n\n### Community and Ecosystem:\nPython has a large and active community, which contributes to the vast ecosystem of libraries, frameworks, and tools. This community support makes it easier to find solutions to problems, learn new techniques, and stay updated with the latest developments in the language.\n\nOverall, Python's versatility, ease of use, and strong community support make it a highly valuable tool for a wide range of programming tasks." additional_kwargs={} response_metadata={'token_usage': {'prompt_tokens': 7, 'total_tokens': 702, 'completion_tokens': 695}, 'model': 'mistral-large-latest', 'finish_reason': 'stop'} id='run-ae9c13b3-2998-4d4b-9ad8-e537968bf57f-0' usage_metadata={'input_tokens': 7, 'output_tokens': 695, 'total_tokens': 702}

Turns out LangChain provides a pretty_print() method for its AIMessage object. And the object returned by the model is an AIMessage object. So…

... ...
llm_out = model.invoke("What is Python? Please be brief.")
# print(llm_out)
llm_out.pretty_print()

(llm-3.13) PS R:\learn\ds_llm> python api_tst.py
================================== Ai Message ==================================

Python is a high-level, interpreted programming language known for its simplicity and readability. It supports multiple programming paradigms, including procedural, object-oriented, and functional programming. Python is widely used for web development, data analysis, artificial intelligence, scientific computing, and more. It's known for its extensive standard library and community support.

And, if I also wanted to print the metadata the above does not contain, it can be obtained as an attribute of the AIMessage object.

... ...
llm_out.pretty_print()
print("\n", llm_out.response_metadata)

(llm-3.13) PS R:\learn\ds_llm> python api_tst.py
================================== Ai Message ==================================

Python is a high-level, interpreted programming language known for its readability and simplicity. It supports multiple programming paradigms, including procedural, object-oriented, and functional programming. Python is widely used for web development, data analysis, artificial intelligence, scientific computing, and automation tasks.

{'token_usage': {'prompt_tokens': 10, 'total_tokens': 70, 'completion_tokens': 60}, 'model': 'mistral-large-latest', 'finish_reason': 'stop'}

Chat History

There is one issue with the simple bot above. It doesn’t keep any history of your conversation with the LLM. Which leaves the LLM in a difficult situation. For example…

... ...
llm_out = model.invoke("My pet's name is Francine.")
llm_out.pretty_print()

llm_out = model.invoke("What is my pet's name?")
llm_out.pretty_print()

(llm-3.13) PS R:\learn\ds_llm> python api_tst.py
================================== Ai Message ==================================

That's a lovely name! What type of pet is Francine? I'd be happy to share some tips or fun facts related to your pet if you'd like.
================================== Ai Message ==================================

I don't have real-time access to information about you or your personal life, including what your pet's name is. If you'd like to share your pet's name, I'd be happy to know it!

There is a basic workaround, pass a growing list of all the individual conversation inputs at each iteration.

... ...
llm_out = model.invoke(["My pet's name is Francine.", "What is my pet's name?"])
llm_out.pretty_print()

(llm-3.13) PS R:\learn\ds_llm> python api_tst.py
================================== Ai Message ==================================

Your pet's name is Francine.

But that hardly seems like fun. Fortunately LangChain and LangGraph can look after that for us. But I really need to do a little more research. So that’s it for today. Maybe for this post—time will tell.

Add Chat History

In a real world applicaton we would likely use a database to store the chat history. But for this simple test we will do that in local memory. So, it will disappear once the app stops; but, not really an issue in this learning context.

There’s another thing I haven’t been able to sort out. All the examples I have seen use Python type annotations in some of the LangGraph method calls. I have not been able to determine if this is strictly necessary. But, everyone must be doing it for some reason. So I will include them as well.

Okay, api test done. So new module, chatbot.py.

I will start with an updated set of imports. I am going to include all the ones I am going to need. But they will only show up in the code as things move along.

Imports

This is the complete list, including those used above.

from typing import Annotated
from typing_extensions import TypedDict

import os, time
from dotenv import load_dotenv

from langchain_mistralai.chat_models import ChatMistralAI

from langchain_core.messages import HumanMessage
from langgraph.checkpoint.memory import MemorySaver
from langgraph.graph import StateGraph, MessagesState, START

StateGraph and Node Definition

We will start by instantiating the graph. Then coding the function that defines our one and only node. If we had more nodes, each one would require a callback function.

StateGraph is the go to for persisting a chat’s state. I will be using it to persistent chat history, but it can be used for other purposes. One of its parameters is the schema class that defines the state to be persisted. As said, in our case, that’s the messages being sent to and received from the LLM. LangGraph provides a schema specifically for this purpose, MessagesState, which we will use.

Only one node in this simple test. It receives the current state as input, does whatever is necessary and returns the updated state.

Getting the api key and instantiating the MistralAI object is unchanged. But from there on things do change.

# Define a new graph
workflow = StateGraph(state_schema=MessagesState)

# Define the function that processes the node, in our case calls the model
# the input parameter state is type annotated as MessagesState
def call_model(state: MessagesState):
    # get llm response to current chat history
    response = model.invoke(state["messages"])
    # Update message history with response
    return {"messages": response}

Add Node and Compile Graph

Okay now we need to add our node to the graph and add an edge from the graph’s entry point to our node. After which we need to compile the graph. To add persistence, we need to pass in a Checkpointer when compiling the graph.

In short: nodes do the work. Edges tell what to do next.
Graphs

# add our single node and an edge to the graph
# the node is labelled as "model" and call_model is added as the callback function
workflow.add_node("model", call_model)
# there are no other nodes, but we need an edge from the special START node to get things going
# i.e. define the entry point to the graph
workflow.add_edge(START, "model")

# compile graph specifying MemorySaver as the checkpointer
mem = MemorySaver()
app = workflow.compile(checkpointer=mem)

I likely should have added an END node to close the graph. A few examples I looked at did so. But, things seem to work without it, so for now I am leaving it out.

Let’s Give It a Go

Another new bit. When running the application, we will pass it a configuration object. Because we are using checkpointer, LangGraph assumes we will have multiple conversations and/or users. This object will specify a thread_id that is used to distinguish different conversational threads. We will call the LangGraph app passing it a user message and the configuration, get the output and display it. We will run a short series of user inputs to see what happens.

# use configuration obj to specify conversation id
cnfg = {"configurable": {"thread_id": "bark1"}}

u_qrys = ["My pet's name is Francine.", "What is my pet's name?", "Francine is a cat.", "What is the best feature of my type of pet? Please be brief."]

for qry in u_qrys:
  in_msgs = [HumanMessage(qry)]
  output = app.invoke({"messages": in_msgs}, cnfg)
  output["messages"][-1].pretty_print()

The first time I tried this I got the following error.

httpx.HTTPStatusError: Error response 429 while fetching https://api.mistral.ai/v1/chat/completions: {"message":"Requests rate limit exceeded"}
During task with name 'model' and id '6bf6a5f4-9188-aad9-96ea-e9306a88f8c6'

So I imported time and added a one second sleep to the loop. Same error. Some searching indicated that the free version of MistralAI la Plateforme LLM has a limit of 1 request per second. So I upped the sleep time to 2 seconds. And that seemed to work just fine. Something I can, for now, live with.

for qry in u_qrys:
  in_msgs = [HumanMessage(qry)]
  output = app.invoke({"messages": in_msgs}, cnfg)
  output["messages"][-1].pretty_print()
  time.sleep(2)

(llm-3.13) PS R:\learn\ds_llm> python chatbot.py
================================== Ai Message ==================================

That's a lovely name! What type of pet is Francine? I'd love to hear more about them.
================================== Ai Message ==================================

You told me your pet's name is Francine.
================================== Ai Message ==================================

Thank you for sharing! How old is Francine and what is she like? Does she have any favorite toys or activities? I'd love to hear more about her.
================================== Ai Message ==================================

One of the best features of having a cat like Francine is their independence and companionship. They can be loving and affectionate while also being content to entertain themselves. Additionally, their playful antics can bring a lot of joy and laughter to your home.

I expected to see my messages displayed along with the LLM’s responses. Not sure if I did something wrong or simply misunderstood what was being saved by the checkpointer. Too lazy to figure it out. Especially as I know future refactoring will do so.

Done

That’s it for this one. More to come. We will look at making the chatbot interactive. And, perhaps adding multiple users to get a feel for the real world.

Resources