LangChain Explained: Your First Steps Toward Building Intelligent Applications with LLMs

Building with large language models can be complex. LangChain makes it simpler. This open-source framework brings together LLMs, data modules, and workflow tools—all in one place—to power up your next AI project.

LangChain is an open-source framework designed to simplify the creation of applications using large language models (LLMs). Whether you're building chatbots, intelligent data retrieval systems, or more complex generative applications, LangChain provides a cohesive environment for combining LLMs with different modules to create powerful workflows. Below, we provide an overview of the important concepts, building blocks, and integrations available within LangChain.

Key Components and Building Blocks of LangChain

LangChain is built around several core packages that serve different purposes:

  • langchain-core: This package contains the base abstractions and interfaces for all the components of LangChain. It defines the structure for core concepts like LLMs, vector stores, retrievers, etc. Importantly, no third-party integrations are included here, ensuring lightweight dependencies.
  • langchain: The main package that contains chains, agents, and retrieval strategies. These components form the "cognitive architecture" for building applications, and are generic across different integrations.
  • langchain-community: This package contains community-maintained third-party integrations, covering LLMs, vector stores, and retrievers.
  • Partner Packages: Popular integrations, like those for OpenAI and Anthropic, are separated into distinct packages (e.g., langchain-openai) for better support.

Additionally, there are specialized extensions such as:

  • LangGraph: Designed for building stateful multi-actor applications, LangGraph uses graph modeling to create sophisticated chains and agents.
  • langserve: A package that helps you deploy LangChain applications as REST APIs for production use.
  • LangSmith: A developer platform that supports debugging, testing, evaluating, and monitoring LLM-based applications.

Core Concepts in LangChain

1. Models: LLMs and Chat Models

LangChain provides integration with multiple LLMs and chat models. These models are used to generate responses based on input prompts. LangChain does not host any models directly but instead integrates with different third-party providers, including:

  • OpenAI (e.g., GPT-3.5, GPT-4)
  • Anthropic (e.g., Claude)
  • Azure OpenAI Service
  • Google Gemini
  • Cohere
  • NVIDIA
  • FireworksAI
  • Groq
  • MistralAI
  • TogetherAI

The chat models accept sequences of messages as input, which allows for more dynamic conversational interactions, distinguishing between roles like user, assistant, and system messages.

2. Prompt Templates

Prompts are the way users communicate instructions to language models. In LangChain, Prompt Templates help convert user input and context into properly formatted prompts that guide the model. Prompt templates can include variables, making it easy to create flexible prompts based on different user inputs.

There are two main types of prompt templates:

  • String Prompt Templates: Used for simpler tasks where the prompt is a single string.
  • Chat Prompt Templates: These are used to format more complex prompts involving multiple messages (e.g., system, user, assistant).

Example:

from langchain_core.prompts import ChatPromptTemplate

prompt_template = ChatPromptTemplate.from_messages([
    ("system", "You are a helpful assistant"),
    ("user", "Tell me a joke about {topic}")
])

3. Chains

Chains are sequences of calls that take user input, process it through models and other tools, and return the result. LangChain provides multiple types of chains:

  • LLMChain: This is the simplest type, consisting of a prompt fed into an LLM.
  • ConversationalRetrievalChain: A more complex chain used for building conversational applications that need context retrieval from past conversations.

4. Agents

Agents are dynamic systems that use an LLM to decide which actions to take next. They form the decision-making backbone of applications that need to interact with tools or APIs based on user inputs.

  • ReAct Agents: These agents use reasoning and acting steps iteratively to complete tasks. For example, they might call a search tool, analyze the results, and decide on the next action.
  • LangGraph Agents: These are more advanced agents aimed at highly controllable and customizable use cases. LangGraph provides the flexibility to compose custom flows using graph-based modeling.

5. Chat History

LangChain provides Chat History functionality, which is crucial for conversational applications. It enables the system to refer back to previous messages, thus maintaining context throughout the conversation.

6. Output Parsers

Output Parsers are used to convert the raw text output from models into structured formats. LangChain supports a variety of parsers, such as:

  • JSON Output Parser: Converts output into a JSON object based on a specified schema.
  • CSV Output Parser: Returns data as a list of comma-separated values.
  • Pandas DataFrame Parser: Converts output into a Pandas DataFrame for easy data manipulation.

Output parsers are especially useful when working with structured data or integrating LLMs with downstream applications.

7. Retrievers and Vector Stores

Retrievers are used to fetch documents based on a query. A Vector Store is a common implementation where documents are embedded into vector representations and then searched using similarity metrics.

  • Popular Vector Stores: LangChain integrates with vector databases like Pinecone, Weaviate, and FAISS, making it easy to set up retrieval-augmented generation (RAG) systems.
  • Retrievers from Vector Stores: You can use vector stores to create retrievers that perform similarity searches and return relevant documents.

8. Tools and Toolkits

Tools are utility functions that an LLM can call to execute specific tasks, such as making an API call or querying a database.

  • Toolkits: A collection of tools designed for specific tasks. For instance, a toolkit might include tools for querying a database, sending an email, or summarizing a document.

LangChain's tools have a name, a description, and a defined schema for inputs, making it easy for the LLM to determine which tool to use in a given context.

LangChain Integrations

LangChain supports many integrations to enhance its capabilities:

  • LLM Integrations: As mentioned earlier, LangChain can integrate with various LLM providers like OpenAI, Anthropic, and Cohere.
  • Document Loaders: These are used to bring data into LangChain from sources like Google Drive, Notion, Slack, and databases.
  • Text Splitters: Text splitters help divide larger documents into smaller, semantically meaningful chunks, making them suitable for LLM processing. For instance, you can split HTML using HTMLHeaderTextSplitter or Markdown with MarkdownHeaderTextSplitter.
  • Key-Value Stores: LangChain also offers key-value stores for use cases like retrieval caching and embeddings management.

LangChain Expression Language (LCEL)

LCEL is a declarative way to chain components together. It provides features like:

  • Streaming: LCEL allows streaming of tokens from an LLM to output parsers, offering fast, real-time user experiences.
  • Async Support: Chains defined with LCEL can be run asynchronously, enabling concurrency and better performance in production environments.
  • Retries and Fallbacks: LCEL supports robust error handling, such as retrying failed requests and configuring fallbacks for different scenarios.

Example LCEL usage:

from langchain_core.prompts import ChatPromptTemplate
from langchain_anthropic import ChatAnthropic

prompt = ChatPromptTemplate.from_template("What's the weather like in {location}?")
model = ChatAnthropic(model="claude-3")
chain = prompt | model

LangChain Packages for Specialized Use Cases

1. LangGraph

LangGraph is aimed at building applications with robust state management. It extends LangChain to enable complex, stateful interactions by modeling the workflow as a graph of nodes and edges. This helps in designing reliable, multi-step agents and defining how data flows between components.

2. LangServe

LangServe makes it easy to deploy LangChain applications as REST APIs. This is particularly useful for developers looking to deploy LLM applications in a production environment without needing to manually manage server infrastructure.

3. LangSmith

LangSmith is a platform for testing, debugging, and monitoring LLM applications. It provides powerful tools for tracking the performance of models, understanding the logic behind their responses, and visualizing how different parts of your chain contribute to the final result.

Putting It All Together

To create a complete LangChain application, you need to:

  1. Choose the right models (e.g., OpenAI's GPT-4 or Anthropic's Claude).
  2. Design a chain or agent that defines how different components (e.g., LLMs, tools, retrievers) interact to achieve your goal.
  3. Define prompts and output parsers to guide the model’s output into the appropriate form.
  4. Use LangServe to deploy your application and LangSmith to monitor and test it.

Example: Building a Simple LLM Application with LCEL

In this quickstart example, we'll show you how to build a simple LLM application that translates text from English into another language. This is a relatively simple LLM application—just a single LLM call plus some prompting. Still, it's a great way to get started with LangChain, as many features can be built with just some prompting and an LLM call!

Setup

To follow along, you'll need to have LangChain installed. You can install it via pip:

pip install langchain

You'll also need an API key for the LLM provider of your choice, such as OpenAI.

Using Language Models

First, let's initialize a language model. In this example, we'll use OpenAI's GPT-4 model.

import os
from langchain_openai import ChatOpenAI

# Set your OpenAI API key
os.environ["OPENAI_API_KEY"] = "your_openai_api_key_here"

# Initialize the model
model = ChatOpenAI(model="gpt-4")

Prompt Templates and Output Parsers

Next, let's define a prompt template and an output parser.

from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser

# Define the prompt template
prompt_template = ChatPromptTemplate.from_messages([
    ("system", "Translate the following into {language}:"),
    ("user", "{text}")
])

# Define the output parser
parser = StrOutputParser()

Chaining Components Together with LCEL

Now, we'll use LCEL to chain the prompt, model, and parser together.

# Create the chain
chain = prompt_template | model | parser

# Invoke the chain
result = chain.invoke({"language": "Italian", "text": "Hello, how are you?"})
print(result)  # Output: 'Ciao, come stai?'

Deploying with LangServe

To deploy this chain as a REST API, you can use LangServe.

from fastapi import FastAPI
from langserve import add_routes

# Define the FastAPI app
app = FastAPI(title="Translation API", version="1.0")

# Add the chain route
add_routes(app, chain, path="/translate")

if __name__ == "__main__":
    import uvicorn
    uvicorn.run(app, host="localhost", port=8000)

You can now run this script to serve your chain at http://localhost:8000/translate.

Final Thoughts

LangChain offers a powerful, flexible framework to build applications powered by language models. With support for different integrations, complex workflows, and robust monitoring tools, it provides all the tools needed to build sophisticated LLM applications. Our simple example shows how you can start building your own applications by chaining components together and deploying them with ease.

‍
Cohorte Team

November 5, 2024