Chapter 15

Week 4 Day 5 - the Sidekick

Notebook.venv31 cells

Week 4 Day 5 - the Sidekick

This is the project the whole week has been building towards. The Sidekick is a personal co-worker you can give a task and a definition of success, and it will work away using a real browser, a filesystem, web search and more, until it has met your criteria or needs to ask you something.

It is also where the stack-not-a-ladder idea pays off. The Sidekick's worker is a single create_agent from Layer 3, but we wrap it in our own loop rather than letting a framework run the show, and we reach down to Layer 1 tools and out to MCP servers as we please. You pick the altitude that suits each part of the job.

How it is built

The worker is one create_agent with all the tools and a stack of middleware:

TodoListMiddleware gives it a plan that it keeps updated as it works, and we surface that plan in the UI.
PIIMiddleware redacts email addresses from the input, and scrubs credit card numbers even out of what the browser sees.
ModelCallLimitMiddleware caps a run at 30 model calls, so a lost agent cannot burn money forever.
HumanInTheLoopMiddleware pauses for your approval before a push notification goes out, and powers the request_human_help tool, which lets the agent hand the browser back to you for logins, captchas and two-factor prompts.
Our own TolerateToolErrors hands tool failures back to the model so it can recover.

Around the worker we run a small loop of our own:

The worker attempts the task.
An evaluator, which is just a model with structured output, checks the answer against your success criteria, with the worker's tool calls as evidence.
If the criteria are met, or the worker has a question, we stop and show you the result. Otherwise we hand the feedback back to the worker and let it try again.

One more thing happens under the hood: the browser and filesystem MCP servers run in persistent sessions held open for the Sidekick's lifetime, so the browser keeps its state between tool calls and can even be handed to a human mid-task. The sessions live in a small background task in sidekick_tools.py, because the stdio transport must be opened and closed from the same task. A welcome side effect is that cleanup() really does close the browser.

We will build up to it in three steps. First the simplest possible version with no evaluator, then human-in-the-loop with middleware, then the full Sidekick from the project modules.

One adjustment for Windows

Same as Day 3: when a stdio MCP server is launched from inside Jupyter on Windows, the kernel's stderr has no real file descriptor and the launch fails. The cell below redirects the servers' error log so everything works. On Mac and Linux it does nothing.

In [ ]python · cell 5

python

import sys

if sys.platform == "win32":
    import subprocess
    from functools import partial
    import langchain_mcp_adapters.sessions as mcp_sessions

    mcp_sessions.stdio_client = partial(mcp_sessions.stdio_client, errlog=subprocess.DEVNULL)
    print("Applied the Windows adjustment")
else:
    print("Not Windows, so nothing to do here")

In [ ]python · cell 6

python

# Imports and environment first

from dotenv import load_dotenv
from langchain.agents import create_agent
from langchain.agents.middleware import HumanInTheLoopMiddleware
from langgraph.checkpoint.memory import InMemorySaver
from langgraph.types import Command

from sidekick_tools import search, send_push_notification, wikipedia_lookup
from sidekick import Sidekick

load_dotenv(override=True)

Step 1: the simplest Sidekick, with no evaluator

At its plainest, the Sidekick is a create_agent with a few tools and a memory, plus a tiny function that runs one turn of conversation. The tools are old friends: the same off-the-shelf Serper search and Wikipedia tools we used earlier in the week, plus our own push notification tool. There is no evaluator here, so the worker simply does its best and replies. For many tasks this simple version is all you need, and everything that follows adds layers on top of it.

In [ ]python · cell 8

python

simple_worker = create_agent(
    model="openai:gpt-5.4-mini",
    tools=[search, send_push_notification, wikipedia_lookup],
    system_prompt="You are Sidekick, a helpful personal assistant. Use your tools to complete the task.",
    checkpointer=InMemorySaver(),
)

simple_worker

In [ ]python · cell 9

python

async def ask(worker, message):
    config = {"configurable": {"thread_id": "simple-sidekick"}}
    result = await worker.ainvoke({"messages": [{"role": "user", "content": message}]}, config=config)
    return result["messages"][-1].content

reply = await ask(simple_worker, "Search for who won the Nobel Prize in Physics in 2023 and send a push notification with a short summary.")
print(reply)

Step 2: human-in-the-loop with middleware

LangChain supports allowing the agent to pause to collect feedback from a user. We tell HumanInTheLoopMiddleware which tools to pause on, and the agent stops and waits for our decision instead of running them.

When the agent reaches such a tool, the run pauses and returns an interrupt that describes the pending action. We can then approve, edit, reject, or respond, and resume.

In [ ]python · cell 11

python

from langchain_core.tools import tool

@tool
def book_meeting(person: str, day: str) -> str:
    """Book a meeting with a person on a given day."""
    return f"Meeting booked with {person} on {day}."

approval_agent = create_agent(
    model="openai:gpt-5.4-mini",
    tools=[book_meeting],
    system_prompt="You are a scheduling assistant. Use the book_meeting tool.",
    middleware=[HumanInTheLoopMiddleware(interrupt_on={"book_meeting": True})],
    checkpointer=InMemorySaver(),
)

config = {"configurable": {"thread_id": "approval-demo"}}
result = await approval_agent.ainvoke(
    {"messages": [{"role": "user", "content": "Book a meeting with Sam on Friday."}]}, config
)

interrupt = result["__interrupt__"][0]
print("The agent paused and is asking for approval:")
print(interrupt.value["action_requests"][0]["description"])

The agent is waiting. We approve the action, and the run continues from exactly where it paused.

Hold this thought: the full Sidekick uses this same middleware to gate its push notifications, and to power a request_human_help tool that lets the agent ask YOU to do something in its browser window, like logging in to a site, and carry on once you have done it.

In [ ]python · cell 13

python

resumed = await approval_agent.ainvoke(Command(resume={"decisions": [{"type": "approve"}]}), config)
print(resumed["messages"][-1].content)

Choosing how to respond

We approved that booking, but approve is only one of the choices. Your resume carries one decision per paused tool call, and each decision can be one of three kinds:

Approve runs the tool exactly as proposed: {"type": "approve"}.
Edit runs it, but with arguments you change first: {"type": "edit", "edited_action": {"name": "book_meeting", "args": {"person": "Sam", "day": "Monday"}}}.
Reject skips the tool and hands a note back to the model so it can try another way: {"type": "reject", "message": "No meetings on Fridays"}.

So to decline the booking rather than approve it, you would resume like this instead:

python

await approval_agent.ainvoke(
    Command(resume={"decisions": [{"type": "reject", "message": "No meetings on Fridays"}]}), config
)

You decide which of these a tool allows when you configure the middleware: interrupt_on={"book_meeting": True} permits all three, while {"book_meeting": {"allowed_decisions": ["approve", "reject"]}} would offer approve or reject but no editing. The Sidekick keeps things to plain approval, but edit and reject are the same one line of resume away.

In [ ]python · cell 15

python

config = {"configurable": {"thread_id": "rejection-demo2"}}
result = await approval_agent.ainvoke(
    {"messages": [{"role": "user", "content": "Book a meeting with Sam on Friday."}]}, config
)

interrupt = result["__interrupt__"][0]
print("The agent paused and is asking for approval:")
print(interrupt.value["action_requests"][0]["description"])

In [ ]python · cell 16

python

resumed = await approval_agent.ainvoke(Command(resume={"decisions": [{"type": "reject", "message": "Rejected - no meetings on Fridays"}]}), config)
print(resumed["messages"][-1].content)

Step 3: the full Sidekick

The complete version lives in sidekick.py and sidekick_tools.py. The worker has the full toolkit: a headed browser and a sandbox filesystem through persistent MCP sessions, plus web search, Wikipedia, push notifications and request_human_help. The middleware stack from the start of this lab is all in place, and the evaluator loop checks each answer against your success criteria, using the worker's tool calls as evidence, sending the worker back to try again when needed.

This is the whole stack in one object. The worker is a Layer 3 create_agent, its tools are Layer 1 @tool functions alongside MCP servers, its memory is a Layer 2 checkpointer, and the loop around it is code you wrote yourself. You chose the altitude for each part.

Let us bring one to life and start with a quick browser errand. Watch the window open and drive itself. The top story on Hacker News changes through the day, so you will see a different headline than the one shown here.

In [ ]python · cell 18

python

sidekick = Sidekick()
await sidekick.setup()
print(f"Sidekick ready with {len(sidekick.tools)} tools")

In [ ]python · cell 19

python

sidekick.tools

In [ ]python · cell 20

python

from langchain_core.runnables.graph_mermaid import draw_mermaid_png
from IPython.display import Image
mm = sidekick.worker.get_graph().draw_mermaid().replace("[", " ").replace("]", " ")
Image(draw_mermaid_png(mm, output_file_path="graph.png"))

In [ ]python · cell 21

python

history = await sidekick.run_turn(
    message="Go to Hacker News at news.ycombinator.com and tell me the title of the current top story.",
    success_criteria="The reply names a specific story currently on the Hacker News front page.",
    history=[],
)
for entry in history:
    print(f"[{entry['role']}] {entry['content'][:200]}\n")

The finale: a real errand

Now a task worthy of the week: find me the best flight. This needs a plan, a stretch of real browsing, a file, and a push notification, and it ends with the Sidekick asking your permission. One detail to notice in sidekick.py: the worker's system prompt carries a practical trick, that Google Flights accepts a natural language query straight in the URL, giving your agent specialist abilities.

While it works, watch the browser window, and peek at the plan: the worker maintains a todo list through its write_todos tool, and the Sidekick exposes it as sidekick.todos.

In [ ]python · cell 23

python

flight_task = """Find me the best round-trip flight from New York to London, leaving about a month from now
and returning a week later. I care about price first, then total journey time, and I would rather avoid
itineraries with two or more stops. Write your recommendation with the top three options to flights.md,
then send me a push notification with the price of your top pick."""

flight_criteria = "flights.md is written with three specific options including airline, times and price, plus a clear recommendation, and a push notification was sent with the recommended price."

history = await sidekick.run_turn(flight_task, flight_criteria, history)
print(history[-1]["content"])

The Sidekick has done the work, and now it is paused, asking permission to send the notification. Its plan tells the story:

In [ ]python · cell 25

python

for todo in sidekick.todos:
    print(f"[{todo['status']}] {todo['content']}")

We approve, the notification arrives on your phone, and the evaluator confirms the criteria were met, with the tool calls as its evidence.

In [ ]python · cell 27

python

if sidekick.paused:
    history = await sidekick.resume(history)
for entry in history[-2:]:
    print(f"[{entry['role']}] {entry['content']}\n")

The Gradio app

app.py wraps all of this in a chat interface: a request box, a success criteria box, the Sidekick's plan updating live beside the chat while it works, and an Approve button that appears whenever it pauses for your say-so. Reset really does close the browser now, since the persistent MCP sessions shut down cleanly.

The look and feel lives in styles.py as a theme, a CSS constant and a JS constant. A Gradio 6 note: these are passed to launch(), not to gr.Blocks() as in older Gradio. Run the app from a terminal with uv run app.py, or launch the very same interface right here:

In [ ]python · cell 29

python

from app import ui, LAUNCH_STYLE

ui.launch(**LAUNCH_STYLE)

Chapter 15

Week 4 Day 5 - the Sidekick

Notebook.venv31 cells

Week 4 Day 5 - the Sidekick

How it is built

The worker is one create_agent with all the tools and a stack of middleware:

TodoListMiddleware gives it a plan that it keeps updated as it works, and we surface that plan in the UI.
PIIMiddleware redacts email addresses from the input, and scrubs credit card numbers even out of what the browser sees.
ModelCallLimitMiddleware caps a run at 30 model calls, so a lost agent cannot burn money forever.
HumanInTheLoopMiddleware pauses for your approval before a push notification goes out, and powers the request_human_help tool, which lets the agent hand the browser back to you for logins, captchas and two-factor prompts.
Our own TolerateToolErrors hands tool failures back to the model so it can recover.

Around the worker we run a small loop of our own:

The worker attempts the task.
An evaluator, which is just a model with structured output, checks the answer against your success criteria, with the worker's tool calls as evidence.
If the criteria are met, or the worker has a question, we stop and show you the result. Otherwise we hand the feedback back to the worker and let it try again.

We will build up to it in three steps. First the simplest possible version with no evaluator, then human-in-the-loop with middleware, then the full Sidekick from the project modules.

One adjustment for Windows

In [ ]python · cell 5

python

import sys

if sys.platform == "win32":
    import subprocess
    from functools import partial
    import langchain_mcp_adapters.sessions as mcp_sessions

    mcp_sessions.stdio_client = partial(mcp_sessions.stdio_client, errlog=subprocess.DEVNULL)
    print("Applied the Windows adjustment")
else:
    print("Not Windows, so nothing to do here")

In [ ]python · cell 6

python

# Imports and environment first

from dotenv import load_dotenv
from langchain.agents import create_agent
from langchain.agents.middleware import HumanInTheLoopMiddleware
from langgraph.checkpoint.memory import InMemorySaver
from langgraph.types import Command

from sidekick_tools import search, send_push_notification, wikipedia_lookup
from sidekick import Sidekick

load_dotenv(override=True)

Step 1: the simplest Sidekick, with no evaluator

In [ ]python · cell 8

python

simple_worker = create_agent(
    model="openai:gpt-5.4-mini",
    tools=[search, send_push_notification, wikipedia_lookup],
    system_prompt="You are Sidekick, a helpful personal assistant. Use your tools to complete the task.",
    checkpointer=InMemorySaver(),
)

simple_worker

In [ ]python · cell 9

python

async def ask(worker, message):
    config = {"configurable": {"thread_id": "simple-sidekick"}}
    result = await worker.ainvoke({"messages": [{"role": "user", "content": message}]}, config=config)
    return result["messages"][-1].content

reply = await ask(simple_worker, "Search for who won the Nobel Prize in Physics in 2023 and send a push notification with a short summary.")
print(reply)

Step 2: human-in-the-loop with middleware

When the agent reaches such a tool, the run pauses and returns an interrupt that describes the pending action. We can then approve, edit, reject, or respond, and resume.

In [ ]python · cell 11

python

from langchain_core.tools import tool

@tool
def book_meeting(person: str, day: str) -> str:
    """Book a meeting with a person on a given day."""
    return f"Meeting booked with {person} on {day}."

approval_agent = create_agent(
    model="openai:gpt-5.4-mini",
    tools=[book_meeting],
    system_prompt="You are a scheduling assistant. Use the book_meeting tool.",
    middleware=[HumanInTheLoopMiddleware(interrupt_on={"book_meeting": True})],
    checkpointer=InMemorySaver(),
)

config = {"configurable": {"thread_id": "approval-demo"}}
result = await approval_agent.ainvoke(
    {"messages": [{"role": "user", "content": "Book a meeting with Sam on Friday."}]}, config
)

interrupt = result["__interrupt__"][0]
print("The agent paused and is asking for approval:")
print(interrupt.value["action_requests"][0]["description"])

The agent is waiting. We approve the action, and the run continues from exactly where it paused.

In [ ]python · cell 13

python

resumed = await approval_agent.ainvoke(Command(resume={"decisions": [{"type": "approve"}]}), config)
print(resumed["messages"][-1].content)

Choosing how to respond

We approved that booking, but approve is only one of the choices. Your resume carries one decision per paused tool call, and each decision can be one of three kinds:

Approve runs the tool exactly as proposed: {"type": "approve"}.
Edit runs it, but with arguments you change first: {"type": "edit", "edited_action": {"name": "book_meeting", "args": {"person": "Sam", "day": "Monday"}}}.
Reject skips the tool and hands a note back to the model so it can try another way: {"type": "reject", "message": "No meetings on Fridays"}.

So to decline the booking rather than approve it, you would resume like this instead:

python

await approval_agent.ainvoke(
    Command(resume={"decisions": [{"type": "reject", "message": "No meetings on Fridays"}]}), config
)

In [ ]python · cell 15

python

config = {"configurable": {"thread_id": "rejection-demo2"}}
result = await approval_agent.ainvoke(
    {"messages": [{"role": "user", "content": "Book a meeting with Sam on Friday."}]}, config
)

interrupt = result["__interrupt__"][0]
print("The agent paused and is asking for approval:")
print(interrupt.value["action_requests"][0]["description"])

In [ ]python · cell 16

python

resumed = await approval_agent.ainvoke(Command(resume={"decisions": [{"type": "reject", "message": "Rejected - no meetings on Fridays"}]}), config)
print(resumed["messages"][-1].content)

Step 3: the full Sidekick

In [ ]python · cell 18

python

sidekick = Sidekick()
await sidekick.setup()
print(f"Sidekick ready with {len(sidekick.tools)} tools")

In [ ]python · cell 19

python

sidekick.tools

In [ ]python · cell 20

python

from langchain_core.runnables.graph_mermaid import draw_mermaid_png
from IPython.display import Image
mm = sidekick.worker.get_graph().draw_mermaid().replace("[", " ").replace("]", " ")
Image(draw_mermaid_png(mm, output_file_path="graph.png"))

In [ ]python · cell 21

python

history = await sidekick.run_turn(
    message="Go to Hacker News at news.ycombinator.com and tell me the title of the current top story.",
    success_criteria="The reply names a specific story currently on the Hacker News front page.",
    history=[],
)
for entry in history:
    print(f"[{entry['role']}] {entry['content'][:200]}\n")

The finale: a real errand

While it works, watch the browser window, and peek at the plan: the worker maintains a todo list through its write_todos tool, and the Sidekick exposes it as sidekick.todos.

In [ ]python · cell 23

python

flight_task = """Find me the best round-trip flight from New York to London, leaving about a month from now
and returning a week later. I care about price first, then total journey time, and I would rather avoid
itineraries with two or more stops. Write your recommendation with the top three options to flights.md,
then send me a push notification with the price of your top pick."""

flight_criteria = "flights.md is written with three specific options including airline, times and price, plus a clear recommendation, and a push notification was sent with the recommended price."

history = await sidekick.run_turn(flight_task, flight_criteria, history)
print(history[-1]["content"])

The Sidekick has done the work, and now it is paused, asking permission to send the notification. Its plan tells the story:

In [ ]python · cell 25

python

for todo in sidekick.todos:
    print(f"[{todo['status']}] {todo['content']}")

We approve, the notification arrives on your phone, and the evaluator confirms the criteria were met, with the tool calls as its evidence.

In [ ]python · cell 27

python

if sidekick.paused:
    history = await sidekick.resume(history)
for entry in history[-2:]:
    print(f"[{entry['role']}] {entry['content']}\n")

The Gradio app

In [ ]python · cell 29

python

from app import ui, LAUNCH_STYLE

ui.launch(**LAUNCH_STYLE)