Chapter 18

agents

Notebookllm-zoomcamp-2026-agents (3.12.1)50 cells

In [1]python · cell 1

python

print(123)

Output

In [2]python · cell 2

python

!wget https://raw.githubusercontent.com/DataTalksClub/llm-zoomcamp/main/01-agentic-rag/code/rag_helper.py
!wget https://raw.githubusercontent.com/DataTalksClub/llm-zoomcamp/main/01-agentic-rag/code/ingest.py

Output

--2026-05-26 09:42:53--  https://raw.githubusercontent.com/DataTalksClub/llm-zoomcamp/main/01-agentic-rag/code/rag_helper.py
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.108.133, 185.199.110.133, 185.199.111.133, ...
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.108.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 2134 (2.1K) [text/plain]
Saving to: ‘rag_helper.py’

rag_helper.py       100%[===================>]   2.08K  --.-KB/s    in 0s      

2026-05-26 09:42:54 (34.7 MB/s) - ‘rag_helper.py’ saved [2134/2134]

--2026-05-26 09:42:54--  https://raw.githubusercontent.com/DataTalksClub/llm-zoomcamp/main/01-agentic-rag/code/ingest.py
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.109.133, 185.199.111.133, 185.199.110.133, ...
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.109.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 738 [text/plain]
Saving to: ‘ingest.py’

ingest.py           100%[===================>]     738  --.-KB/s    in 0s      

2026-05-26 09:42:54 (40.0 MB/s) - ‘ingest.py’ saved [738/738]

In [3]python · cell 3

python

from dotenv import load_dotenv
from openai import OpenAI

load_dotenv()
openai_client = OpenAI()

In [4]python · cell 4

python

from ingest import load_faq_data, build_index

documents = load_faq_data()
index = build_index(documents)

In [5]python · cell 5

python

from rag_helper import RAGBase


instructions = """
You're a course teaching assistant.
Answer the QUESTION based on the CONTEXT from the FAQ database.
Use only the facts from the CONTEXT when answering the QUESTION.
""".strip()

assistant = RAGBase(
    index=index,
    llm_client=openai_client,
    instructions=instructions,
)

In [7]python · cell 6

python

answer = assistant.rag('How do I run Ollama locally?')
print(answer)

Output

To run Ollama locally:

1. Install Ollama from: https://ollama.com/download  
   - macOS: install the `.pkg`
   - Windows: install the `.msi`
   - Linux:
     ```bash
     curl -fsSL https://ollama.com/install.sh | sh
     ```

2. Open a terminal and run:
   ```bash
   ollama run llama3
   ```

This will download the LLaMA 3 model, start it locally, and open a chat-like interface.

3. To test that the local server is running, you can use:
   ```bash
   curl http://localhost:11434
   ```

If needed, you can also install the Python client with:
```bash
pip install ollama
```

In [8]python · cell 7

python

answer = assistant.rag('How do I run Olama locally?')
print(answer)

Output

I don’t have any information in the FAQ context about running Olama locally.

In [11]python · cell 8

python

messages = [
    {'role': 'user', 'content': 'I just discovered the course. Can I join it?'}
]

response = openai_client.responses.create(
    model='gpt-5.4-mini',
    input=messages,
)

response.output_text

Output

'Yes—usually you can, **if enrollment is still open** and the course hasn’t reached capacity.\n\nTo check:\n1. **Look for the course enrollment page** or registration link.\n2. Check whether there’s a **deadline** or **waitlist**.\n3. If it’s through a school, platform, or instructor, **contact them directly** and ask if late enrollment is allowed.\n\nIf you want, I can help you write a short message to ask about joining.'

In [13]python · cell 9

python

def search(query):
    boost_dict = {'question': 3.0, 'section': 0.5}
    filter_dict = {'course': 'llm-zoomcamp'}

    return index.search(
        query,
        num_results=5,
        boost_dict=boost_dict,
        filter_dict=filter_dict
    )

In [17]python · cell 10

python

search_tool = {
    "type": "function",
    'name': 'search',
    'description': 'Search the FAQ database for entries matching the given query.',
    'parameters': {
        "type": "object",
        "properties": {
            'query': {
                "type": "string",
                'description': 'Search query text to look up in the course FAQ.'
            }
        },
        "required": ["query"],
        'additionalProperties': False
    }
}

In [16]python · cell 11

python

response = openai_client.responses.create(
    model='gpt-5.4-mini',
    input=messages,
    tools=[search_tool]
)

In [20]python · cell 12

python

len(response.output)

Output

In [ ]python · cell 13

python

call = response.output[0]

In [22]python · cell 14

python

call

Output

ResponseFunctionToolCall(arguments='{"query":"join course discovered late can I join enroll late join course"}', call_id='call_UpV80e7ba8LoEnDqsjqAfJeC', name='search', type='function_call', id='fc_0f4fa185156188ae006a156dad12e481938c171d34718b305f', namespace=None, status='completed')

In [30]python · cell 15

python

import json

args = json.loads(call.arguments)
args

Output

{'query': 'join course discovered late can I join enroll late join course'}

In [29]python · cell 16

python

call.name

Output

'search'

In [28]python · cell 17

python

results = search(**args)

In [32]python · cell 18

python

result_json = json.dumps(results, indent=2)

In [35]python · cell 19

python

function_call_output = {
    "type": "function_call_output",
    'call_id': call.call_id,
    'output': result_json,
}

In [39]python · cell 20

python

messages.append(call)

In [41]python · cell 21

python

messages.append(function_call_output)

In [42]python · cell 22

python

messages

Output

[{'role': 'user', 'content': 'I just discovered the course. Can I join it?'},
 ResponseFunctionToolCall(arguments='{"query":"join course discovered late can I join enroll late join course"}', call_id='call_UpV80e7ba8LoEnDqsjqAfJeC', name='search', type='function_call', id='fc_0f4fa185156188ae006a156dad12e481938c171d34718b305f', namespace=None, status='completed'),
 {'type': 'function_call_output',
  'call_id': 'call_UpV80e7ba8LoEnDqsjqAfJeC',
  'output': '[\n  {\n    "id": "74eb249bbf",\n    "course": "llm-zoomcamp",\n    "section": "General Course-Related Questions",\n    "question": "I just discovered the course. Can I still join?",\n    "answer": "Yes, but if you want to receive a certificate, you need to submit your project while we\\u2019re still accepting submissions."\n  },\n  {\n    "id": "69d122f12e",\n    "course": "llm-zoomcamp",\n    "section": "General Course-Related Questions",\n    "question": "Certificate: Can I follow the course in a self-paced mode and get a certificate?",\n    "answer": "No, you can only get a certificate if you finish the course with a \\"live\\" cohort.\\n\\nWe don\'t award certificates for the self-paced mode. The reason is you need to peer-review 3 capstone(s) after submitting your project.\\n\\nYou can only peer-review projects at the time the course is running; after the form is closed and the peer-review list is compiled."\n  },\n  {\n    "id": "bd31146b0e",\n    "course": "llm-zoomcamp",\n    "section": "General Course-Related Questions",\n    "question": "When will the course be offered next?",\n    "answer": "Summer 2025."\n  },\n  {\n    "id": "977bf7786c",\n    "course": "llm-zoomcamp",\n    "section": "General Course-Related Questions",\n    "question": "Course: I have registered for the LLM Zoomcamp. When can I expect to receive the confirmation email?",\n    "answer": "You don\'t need it. You\'re accepted. You can also just start learning and submitting homework (while the form is open) without registering. It is not checked against any registered list. Registration is just to gauge interest before the start date."\n  },\n  {\n    "id": "c774542f32",\n    "course": "llm-zoomcamp",\n    "section": "General Course-Related Questions",\n    "question": "Server Error (500) When logging in to course homework using GitHub",\n    "answer": "Additional error text seen:\\n\\n```\\nThird-Party Login Failure\\n\\nAn error occurred while attempting to login via your third-party account.\\n```\\n\\nThe current solution is to use Google or Slack to log in and submit homework answers, as the root cause analysis for the GitHub issue is sporadic and doesn\\u2019t impact all users."\n  }\n]'}]

In [43]python · cell 23

python

response = openai_client.responses.create(
    model='gpt-5.4-mini',
    input=messages,
    tools=[search_tool]
)

In [44]python · cell 24

python

print(response.output_text)

Output

Yes — you can still join the course.

If you want a certificate, though, you need to submit your project while submissions are still open.

In [46]python · cell 25

python

usage = response.usage
usage.input_tokens, usage.output_tokens

Output

(652, 33)

In [48]python · cell 26

python

def calculate_gpt54mini_price(input_tokens, output_tokens):
    # Prices per 1M tokens (example pricing)
    INPUT_PRICE_PER_MILLION = 0.15   # $0.15 / 1M input tokens
    OUTPUT_PRICE_PER_MILLION = 0.60  # $0.60 / 1M output tokens

    input_cost = (input_tokens / 1_000_000) * INPUT_PRICE_PER_MILLION
    output_cost = (output_tokens / 1_000_000) * OUTPUT_PRICE_PER_MILLION

    total_cost = input_cost + output_cost

    return {
        "input_cost": input_cost,
        "output_cost": output_cost,
        "total_cost": total_cost
    }


# Your tokens
result = calculate_gpt54mini_price(652, 33)

print("Total Cost: $", round(result["total_cost"], 8))

Output

Total Cost: $ 0.0001176

In [ ]python · cell 27

python

def make_call(call):
    args = json.loads(call.arguments)

    if call.name == 'search':
        result = search(**args)

    result_json = json.dumps(result, indent=2)

    return {
        "type": "function_call_output",
        'call_id': call.call_id,
        'output': result_json,
    }

In [63]python · cell 28

python

instructions = """
You're a course teaching assistant.
You're given a question from a course student and your task is to answer it.

If you want to look up information, use the search function. 
Use as many keywords from the user question as possible when making first requests.

Make multiple searches.

Try to expand your search by using new keywords
based on the results you get from the search.

At the end, ask if there are other areas that the user wants to explore.
"""

question = 'I just discovered the course. Can I join it?'


messages = [
    {'role': 'developer', 'content': instructions},
    {'role': 'user', 'content': question}
]

In [67]python · cell 29

python

response = openai_client.responses.create(
    model='gpt-5.4-mini',
    input=messages,
    tools=[search_tool]
)

In [ ]python · cell 30

python

messages.extend(response.output)

for item in response.output:
    if item.type == 'function_call':
        print('function_call:', item.name, item.arguments)
        call_output = make_call(item)
        messages.append(call_output)

    elif item.type == 'message':
        print('ASSISTANT:')
        print(item.content[0].text)

Output

Yes, you can still join and follow along.

If you want a certificate, though, you’ll need to submit your project while submissions are still open. Also, certificates are only available for the live cohort, not self-paced.

If you want, I can also help with:
- whether registration is required,
- how homework submission works,
- or when the course runs next.

Is there anything else you’d like to explore?

In [77]python · cell 31

python

messages

Output

[{'role': 'developer',
  'content': "\nYou're a course teaching assistant.\nYou're given a question from a course student and your task is to answer it.\n\nIf you want to look up information, use the search function. \nUse as many keywords from the user question as possible when making first requests.\n\nMake multiple searches.\n\nTry to expand your search by using new keywords\nbased on the results you get from the search.\n\nAt the end, ask if there are other areas that the user wants to explore.\n"},
 {'role': 'user', 'content': 'I just discovered the course. Can I join it?'},
 ResponseFunctionToolCall(arguments='{"query":"join course discovered the course can I join enrollment registration"}', call_id='call_6SVouYKfkyyId3q1pVx8CsJW', name='search', type='function_call', id='fc_008dbefb78b869a2006a1574253d7881928dd1ebd1d2eebe58', namespace=None, status='completed'),
 ResponseFunctionToolCall(arguments='{"query":"course access enrollment late join new student can I join"}', call_id='call_Od8fzAaFVAJ7fKgKfqUSx6rv', name='search', type='function_call', id='fc_008dbefb78b869a2006a1574253d88819284770e1a20aee266', namespace=None, status='completed'),
 {'type': 'function_call_output',
  'call_id': 'call_6SVouYKfkyyId3q1pVx8CsJW',
  'output': '[\n  {\n    "id": "74eb249bbf",\n    "course": "llm-zoomcamp",\n    "section": "General Course-Related Questions",\n    "question": "I just discovered the course. Can I still join?",\n    "answer": "Yes, but if you want to receive a certificate, you need to submit your project while we\\u2019re still accepting submissions."\n  },\n  {\n    "id": "69d122f12e",\n    "course": "llm-zoomcamp",\n    "section": "General Course-Related Questions",\n    "question": "Certificate: Can I follow the course in a self-paced mode and get a certificate?",\n    "answer": "No, you can only get a certificate if you finish the course with a \\"live\\" cohort.\\n\\nWe don\'t award certificates for the self-paced mode. The reason is you need to peer-review 3 capstone(s) after submitting your project.\\n\\nYou can only peer-review projects at the time the course is running; after the form is closed and the peer-review list is compiled."\n  },\n  {\n    "id": "977bf7786c",\n    "course": "llm-zoomcamp",\n    "section": "General Course-Related Questions",\n    "question": "Course: I have registered for the LLM Zoomcamp. When can I expect to receive the confirmation email?",\n    "answer": "You don\'t need it. You\'re accepted. You can also just start learning and submitting homework (while the form is open) without registering. It is not checked against any registered list. Registration is just to gauge interest before the start date."\n  },\n  {\n    "id": "bd31146b0e",\n    "course": "llm-zoomcamp",\n    "section": "General Course-Related Questions",\n    "question": "When will the course be offered next?",\n    "answer": "Summer 2025."\n  },\n  {\n    "id": "c774542f32",\n    "course": "llm-zoomcamp",\n    "section": "General Course-Related Questions",\n    "question": "Server Error (500) When logging in to course homework using GitHub",\n    "answer": "Additional error text seen:\\n\\n```\\nThird-Party Login Failure\\n\\nAn error occurred while attempting to login via your third-party account.\\n```\\n\\nThe current solution is to use Google or Slack to log in and submit homework answers, as the root cause analysis for the GitHub issue is sporadic and doesn\\u2019t impact all users."\n  }\n]'},
 {'type': 'function_call_output',
  'call_id': 'call_Od8fzAaFVAJ7fKgKfqUSx6rv',
  'output': '[\n  {\n    "id": "74eb249bbf",\n    "course": "llm-zoomcamp",\n    "section": "General Course-Related Questions",\n    "question": "I just discovered the course. Can I still join?",\n    "answer": "Yes, but if you want to receive a certificate, you need to submit your project while we\\u2019re still accepting submissions."\n  },\n  {\n    "id": "69d122f12e",\n    "course": "llm-zoomcamp",\n    "section": "General Course-Related Questions",\n    "question": "Certificate: Can I follow the course in a self-paced mode and get a certificate?",\n    "answer": "No, you can only get a certificate if you finish the course with a \\"live\\" cohort.\\n\\nWe don\'t award certificates for the self-paced mode. The reason is you need to peer-review 3 capstone(s) after submitting your project.\\n\\nYou can only peer-review projects at the time the course is running; after the form is closed and the peer-review list is compiled."\n  },\n  {\n    "id": "977bf7786c",\n    "course": "llm-zoomcamp",\n    "section": "General Course-Related Questions",\n    "question": "Course: I have registered for the LLM Zoomcamp. When can I expect to receive the confirmation email?",\n    "answer": "You don\'t need it. You\'re accepted. You can also just start learning and submitting homework (while the form is open) without registering. It is not checked against any registered list. Registration is just to gauge interest before the start date."\n  },\n  {\n    "id": "bd31146b0e",\n    "course": "llm-zoomcamp",\n    "section": "General Course-Related Questions",\n    "question": "When will the course be offered next?",\n    "answer": "Summer 2025."\n  },\n  {\n    "id": "6bceeebf04",\n    "course": "llm-zoomcamp",\n    "section": "Workshop: Open-Source Data Ingestion (dlt)",\n    "question": "How to set up a new dlt project when loading from cloud?",\n    "answer": "Start with the following command on the command line:\\n\\n```bash\\n dlt init filesystem duckdb\\n```\\n\\nMore directions can be found at [dlthub.com](https://dlthub.com/docs/tutorial/filesystem)"\n  }\n]'},
 ResponseOutputMessage(id='msg_008dbefb78b869a2006a15743171708192ae736e385a88251e', content=[ResponseOutputText(annotations=[], text='Yes, you can still join and follow along.\n\nIf you want a certificate, though, you’ll need to submit your project while submissions are still open. Also, certificates are only available for the live cohort, not self-paced.\n\nIf you want, I can also help with:\n- whether registration is required,\n- how homework submission works,\n- or when the course runs next.\n\nIs there anything else you’d like to explore?', type='output_text', logprobs=[])], role='assistant', status='completed', type='message', phase='final_answer')]

In [79]python · cell 32

python

messages = [
    {'role': 'developer', 'content': instructions},
    {'role': 'user', 'content': question}
]

it = 1

while True:
    print(f'iteration #{it}...')
    has_function_calls = False

    response = openai_client.responses.create(
        model='gpt-5.4-mini',
        input=messages,
        tools=[search_tool]
    )

    messages.extend(response.output)

    for item in response.output:
        if item.type == 'function_call':
            print('function_call:', item.name, item.arguments)
            call_output = make_call(item)
            messages.append(call_output)
            has_function_calls = True

        elif item.type == 'message':
            print('ASSISTANT:')
            print(item.content[0].text)
    
    it = it + 1
    if has_function_calls == False:
        break

Output

iteration #1...
function_call: search {"query":"join course discovered course can I join enrollment late joining FAQ"}
function_call: search {"query":"new student can I join course after start FAQ enrollment"}
function_call: search {"query":"course access enrollment deadline can join FAQ"}
iteration #2...
ASSISTANT:
Yes — you can still join the course.

If you want a certificate, make sure to submit your project while submissions are still open. Also, certificates are only available for the live cohort, not self-paced participation.

If you’d like, I can also help with how registration, homework submission, or certificates work.

In [87]python · cell 33

python

def agent_loop(instructions, question, model='gpt-5.4-mini') -> str:
    messages = [
        {'role': 'developer', 'content': instructions},
        {'role': 'user', 'content': question}
    ]

    it = 1

    while True:
        print(f'iteration #{it}...')
        has_function_calls = False

        response = openai_client.responses.create(
            model=model,
            input=messages,
            tools=[search_tool]
        )

        messages.extend(response.output)

        for item in response.output:
            if item.type == 'function_call':
                print('function_call:', item.name, item.arguments)
                call_output = make_call(item)
                messages.append(call_output)
                has_function_calls = True

            elif item.type == 'message':
                print('ASSISTANT:')
                last_answer = item.content[0].text
                print(item.content[0].text)

        it = it + 1
        if has_function_calls == False:
            break
    
    return last_answer

In [88]python · cell 34

python

instructions = """
You're a course teaching assistant.
You're given a question from a course student and your task is to answer it.

If you want to look up information, use the search function. 
Use as many keywords from the user question as possible when making first requests.

Make multiple searches. First perform search, analyze the results 
and then perform more searchers. 

At the end, ask if there are other areas that the user wants to explore.
"""

question = 'I just discovered the course. Can I join it?'

In [90]python · cell 35

python

result = agent_loop(instructions, question)

Output

iteration #1...
function_call: search {"query":"join course discovered course can I join enrollment registration access FAQ"}
iteration #2...
function_call: search {"query":"self-paced join course certificate live cohort project submission accepted while form open FAQ"}
iteration #3...
ASSISTANT:
Yes — you can still join the course.

If you want a certificate, make sure to submit your project while submissions are still being accepted. If you’re only interested in learning, you can start even if you discovered the course later.

If you want, I can also tell you what’s needed to get the certificate or how the course works.

In [91]python · cell 36

python

result

Output

'Yes — you can still join the course.\n\nIf you want a certificate, make sure to submit your project while submissions are still being accepted. If you’re only interested in learning, you can start even if you discovered the course later.\n\nIf you want, I can also tell you what’s needed to get the certificate or how the course works.'

In [93]python · cell 37

python

instructions = """
You're a course teaching assistant.
You're given a question from a course student and your task is to answer it.

If you want to look up information, use the search function. 
Use as many keywords from the user question as possible when making first requests.

Make multiple searches. First perform search, analyze the results 
and then perform more searchers. 

The question has to be about the course or its logistics, offtopic questions 
shouldn't be answered. If the search returns nothing, it's likely an off-topic question.
If you can't answer the question using FAQ, don't do it yourself. Only use the 
facts from the FAQ database.

At the end, ask if there are other areas that the user wants to explore.
"""

question = "what's queen gambit?"

result = agent_loop(instructions, question)

Output

iteration #1...

function_call: search {"query":"queen gambit"}
iteration #2...
function_call: search {"query":"gambit chess opening queen's gambit course FAQ"}
iteration #3...
ASSISTANT:
I couldn’t find a course FAQ entry for “queen gambit,” so it looks like this may be off-topic for the course.

If you meant **“Queen’s Gambit”** in chess, I can’t answer outside the course FAQ. If you meant something course-related and can rephrase it, I’m happy to help.

Is there anything else in the course or its logistics you’d like to explore?

In [95]python · cell 38

python

from toyaikit.llm import OpenAIClient
from toyaikit.tools import Tools
from toyaikit.chat import IPythonChatInterface
from toyaikit.chat.runners import OpenAIResponsesRunner, DisplayingRunnerCallback

In [96]python · cell 39

python

agent_tools = Tools()
agent_tools.add_tool(search, search_tool)

In [99]python · cell 40

python

def search(query: str) -> dict[str, str]:
    """
    Search the FAQ database for entries matching the given query.
    """
    return index.search(
        query,
        num_results=5,
        boost_dict={'question': 3.0, 'section': 0.5},
        filter_dict={'course': 'llm-zoomcamp'}
    )

In [100]python · cell 41

python

agent_tools = Tools()
agent_tools.add_tool(search)

In [101]python · cell 42

python

agent_tools.get_tools()

Output

[{'type': 'function',
  'name': 'search',
  'description': 'Search the FAQ database for entries matching the given query.',
  'parameters': {'type': 'object',
   'properties': {'query': {'type': 'string',
     'description': 'query parameter'}},
   'required': ['query'],
   'additionalProperties': False}}]

In [105]python · cell 43

python

chat_interface = IPythonChatInterface()
callback = DisplayingRunnerCallback(chat_interface)

In [104]python · cell 44

python

runner = OpenAIResponsesRunner(
    tools=agent_tools,
    developer_prompt=instructions,
    chat_interface=chat_interface,
    llm_client=OpenAIClient(model='gpt-5.4-mini')
)

In [107]python · cell 45

python

result = runner.loop(
    prompt='How do I run Olama locally?',
    callback=callback,
)

Output

-> Response received

<IPython.core.display.HTML object>

Function call: search({"query":"Olama locally run install local FAQ"})

Call

{"query":"Olama locally run install local FAQ"}

Output

[
  {
    "id": "cf93377279",
    "course": "llm-zoomcamp",
    "section": "Module 1: Introduction to LLMs and RAG",
    "question": "ModuleNotFoundError on import docx in parse-faq.ipynb",
    "answer": "The correct package name for `docx` is `python-docx`, not `docx`. Make sure to install the package using:\n\n```bash\npip install python-docx\n```"
  },
  {
    "id": "d09e8d4843",
    "course": "llm-zoomcamp",
    "section": "Module 2: Agents",
    "question": "Install MCP Inspector",
    "answer": "1. Ensure Node.js is installed.\n\n2. To install the MCP Inspector, run the following command in your terminal:\n\n   ```bash\n   npm i @modelcontextprotocol/inspector\n   ```"
  },
  {
    "id": "1d0b969028",
    "course": "llm-zoomcamp",
    "section": "Module 1: Introduction to LLMs and RAG",
    "question": "Ollama: How to install Ollama?",
    "answer": "First, install Ollama by visiting [https://ollama.com/download](https://ollama.com/download) and choosing your operating system:\n\n- **macOS**: Download the `.pkg` and install it.\n- **Windows**: Download the `.msi` and install it.\n- **Linux**: Run the following command in the terminal:\n\n  ```bash\n  curl -fsSL https://ollama.com/install.sh | sh\n  ```\n\nOnce installed, open a terminal and type:\n\n```bash\nollama run llama3\n```\n\nThis command will:\n\n- Download the LLaMA 3 model (~4GB).\n- Start the model locally.\n- Open a chat-like interface where you can type questions.\n\nTo test the Ollama local server, run the following command:\n\n```bash\ncurl http://localhost:11434\n```\n\nYou should receive a response similar to:\n\n```json\n{\"models\": [...]}  \n```\n\nThen, install the Python client with:\n\n```bash\npip install ollama\n```\n\nHere is a minimal Python example:\n\n```python\nimport ollama\n\nresponse = ollama.chat(\n    model='llama3',\n    messages=[{\"role\": \"user\", \"content\": your_prompt}]\n)\n\nprint(response['message']['content'])\n```"
  },
  {
    "id": "42c7af7ed5",
    "course": "llm-zoomcamp",
    "section": "Module 2: Agents",
    "question": "Run MCP Inspector",
    "answer": "To run the MCP Inspector, execute the following command in the terminal:\n\n```bash\nnpx @modelcontextprotocol/inspector\n```"
  },
  {
    "id": "15e931476c",
    "course": "llm-zoomcamp",
    "section": "Module 5: Monitoring",
    "question": "In Windows OS: OSError: [WinError 126] The specified module could not be found. Error loading \"C:\\Users\\USER\\AppData\\Local\\Programs\\Python\\Python310\\lib\\site-packages\\torch\\lib\\fbgemm.dll\" or one of its dependencies.",
    "answer": "- **Solution 1**: Install Visual C++ Redistributable.\n\n- **Solution 2**: Install Visual Studio, not Visual Studio Code. \n\n  <{IMAGE:image_1}>\n\n  For more details, please follow this link: [discuss.pytorch.org](https://discuss.pytorch.org/t/failed-to-import-pytorch-fbgemm-dll-or-one-of-its-dependencies-is-missing/201969)"
  }
]

-> Response received

<IPython.core.display.HTML object>

Function call: search({"query":"Ollama run llama3 local server localh...)

Call

{"query":"Ollama run llama3 local server localhost:11434 FAQ"}

Output

[
  {
    "id": "1d0b969028",
    "course": "llm-zoomcamp",
    "section": "Module 1: Introduction to LLMs and RAG",
    "question": "Ollama: How to install Ollama?",
    "answer": "First, install Ollama by visiting [https://ollama.com/download](https://ollama.com/download) and choosing your operating system:\n\n- **macOS**: Download the `.pkg` and install it.\n- **Windows**: Download the `.msi` and install it.\n- **Linux**: Run the following command in the terminal:\n\n  ```bash\n  curl -fsSL https://ollama.com/install.sh | sh\n  ```\n\nOnce installed, open a terminal and type:\n\n```bash\nollama run llama3\n```\n\nThis command will:\n\n- Download the LLaMA 3 model (~4GB).\n- Start the model locally.\n- Open a chat-like interface where you can type questions.\n\nTo test the Ollama local server, run the following command:\n\n```bash\ncurl http://localhost:11434\n```\n\nYou should receive a response similar to:\n\n```json\n{\"models\": [...]}  \n```\n\nThen, install the Python client with:\n\n```bash\npip install ollama\n```\n\nHere is a minimal Python example:\n\n```python\nimport ollama\n\nresponse = ollama.chat(\n    model='llama3',\n    messages=[{\"role\": \"user\", \"content\": your_prompt}]\n)\n\nprint(response['message']['content'])\n```"
  },
  {
    "id": "9c2fd5451c",
    "course": "llm-zoomcamp",
    "section": "Module 2: Agents",
    "question": "Inspect MCP Server",
    "answer": "- Connect to the MCP Server\n\n  <{IMAGE:image_1}>\n\n- The inspector can list tools, templates, resources, and prompts from the MCP Server\n\n  <{IMAGE:image_2}>\n\nReference:\n\n[https://medium.com/@anil.goyal0057/how-to-test-your-mcp-server-using-mcp-inspector-c873c417eec1](https://medium.com/@anil.goyal0057/how-to-test-your-mcp-server-using-mcp-inspector-c873c417eec1)"
  },
  {
    "id": "cf93377279",
    "course": "llm-zoomcamp",
    "section": "Module 1: Introduction to LLMs and RAG",
    "question": "ModuleNotFoundError on import docx in parse-faq.ipynb",
    "answer": "The correct package name for `docx` is `python-docx`, not `docx`. Make sure to install the package using:\n\n```bash\npip install python-docx\n```"
  },
  {
    "id": "42c7af7ed5",
    "course": "llm-zoomcamp",
    "section": "Module 2: Agents",
    "question": "Run MCP Inspector",
    "answer": "To run the MCP Inspector, execute the following command in the terminal:\n\n```bash\nnpx @modelcontextprotocol/inspector\n```"
  },
  {
    "id": "c774542f32",
    "course": "llm-zoomcamp",
    "section": "General Course-Related Questions",
    "question": "Server Error (500) When logging in to course homework using GitHub",
    "answer": "Additional error text seen:\n\n```\nThird-Party Login Failure\n\nAn error occurred while attempting to login via your third-party account.\n```\n\nThe current solution is to use Google or Slack to log in and submit homework answers, as the root cause analysis for the GitHub issue is sporadic and doesn\u2019t impact all users."
  }
]

-> Response received

<IPython.core.display.HTML object>

Assistant:

To run Ollama locally:

Install Ollama from: https://ollama.com/download
- macOS: download and install the .pkg
- Windows: download and install the .msi
- Linux: run
```
curl -fsSL https://ollama.com/install.sh | sh
```
Start a model in your terminal:
```
ollama run llama3
```
This downloads the model, starts it locally, and opens a chat-like interface.
To test that the local server is running:
```
curl http://localhost:11434
```

If you want to use it from Python, install the client:

pip install ollama

and then use:

import ollama

response = ollama.chat(
    model='llama3',
    messages=[{"role": "user", "content": your_prompt}]
)

print(response['message']['content'])

If you'd like, I can also help with how to run a different model or connect Ollama from a notebook.

In [109]python · cell 46

python

result.cost

Output

CostInfo(input_cost=Decimal('0.002517'), output_cost=Decimal('0.0013005'), total_cost=Decimal('0.0038175'))

In [112]python · cell 47

python

result.all_messages

Output

[EasyInputMessage(content="\nYou're a course teaching assistant.\nYou're given a question from a course student and your task is to answer it.\n\nIf you want to look up information, use the search function. \nUse as many keywords from the user question as possible when making first requests.\n\nMake multiple searches. First perform search, analyze the results \nand then perform more searchers. \n\nThe question has to be about the course or its logistics, offtopic questions \nshouldn't be answered. If the search returns nothing, it's likely an off-topic question.\nIf you can't answer the question using FAQ, don't do it yourself. Only use the \nfacts from the FAQ database.\n\nAt the end, ask if there are other areas that the user wants to explore.\n", role='developer', phase=None, type=None),
 EasyInputMessage(content='How do I run Olama locally?', role='user', phase=None, type=None),
 ResponseFunctionToolCall(arguments='{"query":"Olama locally run install local FAQ"}', call_id='call_uQMD3EKYgvzrP1lijpjqVkG1', name='search', type='function_call', id='fc_09b53c21ed00cd32006a157a290eac8195b8a344b9f7e6558a', namespace=None, status='completed'),
 {'type': 'function_call_output',
  'call_id': 'call_uQMD3EKYgvzrP1lijpjqVkG1',
  'output': '[\n  {\n    "id": "cf93377279",\n    "course": "llm-zoomcamp",\n    "section": "Module 1: Introduction to LLMs and RAG",\n    "question": "ModuleNotFoundError on import docx in parse-faq.ipynb",\n    "answer": "The correct package name for `docx` is `python-docx`, not `docx`. Make sure to install the package using:\\n\\n```bash\\npip install python-docx\\n```"\n  },\n  {\n    "id": "d09e8d4843",\n    "course": "llm-zoomcamp",\n    "section": "Module 2: Agents",\n    "question": "Install MCP Inspector",\n    "answer": "1. Ensure Node.js is installed.\\n\\n2. To install the MCP Inspector, run the following command in your terminal:\\n\\n   ```bash\\n   npm i @modelcontextprotocol/inspector\\n   ```"\n  },\n  {\n    "id": "1d0b969028",\n    "course": "llm-zoomcamp",\n    "section": "Module 1: Introduction to LLMs and RAG",\n    "question": "Ollama: How to install Ollama?",\n    "answer": "First, install Ollama by visiting [https://ollama.com/download](https://ollama.com/download) and choosing your operating system:\\n\\n- **macOS**: Download the `.pkg` and install it.\\n- **Windows**: Download the `.msi` and install it.\\n- **Linux**: Run the following command in the terminal:\\n\\n  ```bash\\n  curl -fsSL https://ollama.com/install.sh | sh\\n  ```\\n\\nOnce installed, open a terminal and type:\\n\\n```bash\\nollama run llama3\\n```\\n\\nThis command will:\\n\\n- Download the LLaMA 3 model (~4GB).\\n- Start the model locally.\\n- Open a chat-like interface where you can type questions.\\n\\nTo test the Ollama local server, run the following command:\\n\\n```bash\\ncurl http://localhost:11434\\n```\\n\\nYou should receive a response similar to:\\n\\n```json\\n{\\"models\\": [...]}  \\n```\\n\\nThen, install the Python client with:\\n\\n```bash\\npip install ollama\\n```\\n\\nHere is a minimal Python example:\\n\\n```python\\nimport ollama\\n\\nresponse = ollama.chat(\\n    model=\'llama3\',\\n    messages=[{\\"role\\": \\"user\\", \\"content\\": your_prompt}]\\n)\\n\\nprint(response[\'message\'][\'content\'])\\n```"\n  },\n  {\n    "id": "42c7af7ed5",\n    "course": "llm-zoomcamp",\n    "section": "Module 2: Agents",\n    "question": "Run MCP Inspector",\n    "answer": "To run the MCP Inspector, execute the following command in the terminal:\\n\\n```bash\\nnpx @modelcontextprotocol/inspector\\n```"\n  },\n  {\n    "id": "15e931476c",\n    "course": "llm-zoomcamp",\n    "section": "Module 5: Monitoring",\n    "question": "In Windows OS: OSError: [WinError 126] The specified module could not be found. Error loading \\"C:\\\\Users\\\\USER\\\\AppData\\\\Local\\\\Programs\\\\Python\\\\Python310\\\\lib\\\\site-packages\\\\torch\\\\lib\\\\fbgemm.dll\\" or one of its dependencies.",\n    "answer": "- **Solution 1**: Install Visual C++ Redistributable.\\n\\n- **Solution 2**: Install Visual Studio, not Visual Studio Code. \\n\\n  <{IMAGE:image_1}>\\n\\n  For more details, please follow this link: [discuss.pytorch.org](https://discuss.pytorch.org/t/failed-to-import-pytorch-fbgemm-dll-or-one-of-its-dependencies-is-missing/201969)"\n  }\n]'},
 ResponseFunctionToolCall(arguments='{"query":"Ollama run llama3 local server localhost:11434 FAQ"}', call_id='call_7wA6k4lXW78oODsDNKKKAgK1', name='search', type='function_call', id='fc_09b53c21ed00cd32006a157a2a94108195a68a9890f34d8b32', namespace=None, status='completed'),
 {'type': 'function_call_output',
  'call_id': 'call_7wA6k4lXW78oODsDNKKKAgK1',
  'output': '[\n  {\n    "id": "1d0b969028",\n    "course": "llm-zoomcamp",\n    "section": "Module 1: Introduction to LLMs and RAG",\n    "question": "Ollama: How to install Ollama?",\n    "answer": "First, install Ollama by visiting [https://ollama.com/download](https://ollama.com/download) and choosing your operating system:\\n\\n- **macOS**: Download the `.pkg` and install it.\\n- **Windows**: Download the `.msi` and install it.\\n- **Linux**: Run the following command in the terminal:\\n\\n  ```bash\\n  curl -fsSL https://ollama.com/install.sh | sh\\n  ```\\n\\nOnce installed, open a terminal and type:\\n\\n```bash\\nollama run llama3\\n```\\n\\nThis command will:\\n\\n- Download the LLaMA 3 model (~4GB).\\n- Start the model locally.\\n- Open a chat-like interface where you can type questions.\\n\\nTo test the Ollama local server, run the following command:\\n\\n```bash\\ncurl http://localhost:11434\\n```\\n\\nYou should receive a response similar to:\\n\\n```json\\n{\\"models\\": [...]}  \\n```\\n\\nThen, install the Python client with:\\n\\n```bash\\npip install ollama\\n```\\n\\nHere is a minimal Python example:\\n\\n```python\\nimport ollama\\n\\nresponse = ollama.chat(\\n    model=\'llama3\',\\n    messages=[{\\"role\\": \\"user\\", \\"content\\": your_prompt}]\\n)\\n\\nprint(response[\'message\'][\'content\'])\\n```"\n  },\n  {\n    "id": "9c2fd5451c",\n    "course": "llm-zoomcamp",\n    "section": "Module 2: Agents",\n    "question": "Inspect MCP Server",\n    "answer": "- Connect to the MCP Server\\n\\n  <{IMAGE:image_1}>\\n\\n- The inspector can list tools, templates, resources, and prompts from the MCP Server\\n\\n  <{IMAGE:image_2}>\\n\\nReference:\\n\\n[https://medium.com/@anil.goyal0057/how-to-test-your-mcp-server-using-mcp-inspector-c873c417eec1](https://medium.com/@anil.goyal0057/how-to-test-your-mcp-server-using-mcp-inspector-c873c417eec1)"\n  },\n  {\n    "id": "cf93377279",\n    "course": "llm-zoomcamp",\n    "section": "Module 1: Introduction to LLMs and RAG",\n    "question": "ModuleNotFoundError on import docx in parse-faq.ipynb",\n    "answer": "The correct package name for `docx` is `python-docx`, not `docx`. Make sure to install the package using:\\n\\n```bash\\npip install python-docx\\n```"\n  },\n  {\n    "id": "42c7af7ed5",\n    "course": "llm-zoomcamp",\n    "section": "Module 2: Agents",\n    "question": "Run MCP Inspector",\n    "answer": "To run the MCP Inspector, execute the following command in the terminal:\\n\\n```bash\\nnpx @modelcontextprotocol/inspector\\n```"\n  },\n  {\n    "id": "c774542f32",\n    "course": "llm-zoomcamp",\n    "section": "General Course-Related Questions",\n    "question": "Server Error (500) When logging in to course homework using GitHub",\n    "answer": "Additional error text seen:\\n\\n```\\nThird-Party Login Failure\\n\\nAn error occurred while attempting to login via your third-party account.\\n```\\n\\nThe current solution is to use Google or Slack to log in and submit homework answers, as the root cause analysis for the GitHub issue is sporadic and doesn\\u2019t impact all users."\n  }\n]'},
 ResponseOutputMessage(id='msg_09b53c21ed00cd32006a157a2c226081958a093fd221d7b228', content=[ResponseOutputText(annotations=[], text='To run Ollama locally:\n\n1. Install Ollama from: https://ollama.com/download  \n   - macOS: download and install the `.pkg`\n   - Windows: download and install the `.msi`\n   - Linux: run\n     ```bash\n     curl -fsSL https://ollama.com/install.sh | sh\n     ```\n\n2. Start a model in your terminal:\n   ```bash\n   ollama run llama3\n   ```\n   This downloads the model, starts it locally, and opens a chat-like interface.\n\n3. To test that the local server is running:\n   ```bash\n   curl http://localhost:11434\n   ```\n\nIf you want to use it from Python, install the client:\n```bash\npip install ollama\n```\n\nand then use:\n```python\nimport ollama\n\nresponse = ollama.chat(\n    model=\'llama3\',\n    messages=[{"role": "user", "content": your_prompt}]\n)\n\nprint(response[\'message\'][\'content\'])\n```\n\nIf you\'d like, I can also help with how to run a different model or connect Ollama from a notebook.', type='output_text', logprobs=[])], role='assistant', status='completed', type='message', phase='final_answer')]

In [114]python · cell 48

python

result2 = runner.loop(
    prompt='How do I run a different model?',
    previous_messages=result.all_messages,
    callback=callback,
)

Output

-> Response received

<IPython.core.display.HTML object>

Function call: search({"query":"Ollama different model run llama3 FAQ...)

Call

{"query":"Ollama different model run llama3 FAQ model name ollama run"}

Output

[
  {
    "id": "1d0b969028",
    "course": "llm-zoomcamp",
    "section": "Module 1: Introduction to LLMs and RAG",
    "question": "Ollama: How to install Ollama?",
    "answer": "First, install Ollama by visiting [https://ollama.com/download](https://ollama.com/download) and choosing your operating system:\n\n- **macOS**: Download the `.pkg` and install it.\n- **Windows**: Download the `.msi` and install it.\n- **Linux**: Run the following command in the terminal:\n\n  ```bash\n  curl -fsSL https://ollama.com/install.sh | sh\n  ```\n\nOnce installed, open a terminal and type:\n\n```bash\nollama run llama3\n```\n\nThis command will:\n\n- Download the LLaMA 3 model (~4GB).\n- Start the model locally.\n- Open a chat-like interface where you can type questions.\n\nTo test the Ollama local server, run the following command:\n\n```bash\ncurl http://localhost:11434\n```\n\nYou should receive a response similar to:\n\n```json\n{\"models\": [...]}  \n```\n\nThen, install the Python client with:\n\n```bash\npip install ollama\n```\n\nHere is a minimal Python example:\n\n```python\nimport ollama\n\nresponse = ollama.chat(\n    model='llama3',\n    messages=[{\"role\": \"user\", \"content\": your_prompt}]\n)\n\nprint(response['message']['content'])\n```"
  },
  {
    "id": "42c7af7ed5",
    "course": "llm-zoomcamp",
    "section": "Module 2: Agents",
    "question": "Run MCP Inspector",
    "answer": "To run the MCP Inspector, execute the following command in the terminal:\n\n```bash\nnpx @modelcontextprotocol/inspector\n```"
  },
  {
    "id": "cf93377279",
    "course": "llm-zoomcamp",
    "section": "Module 1: Introduction to LLMs and RAG",
    "question": "ModuleNotFoundError on import docx in parse-faq.ipynb",
    "answer": "The correct package name for `docx` is `python-docx`, not `docx`. Make sure to install the package using:\n\n```bash\npip install python-docx\n```"
  },
  {
    "id": "c6fc2d4d11",
    "course": "llm-zoomcamp",
    "section": "Module 2: Agents",
    "question": "Connection refused error on prompting the ollam RAG?",
    "answer": "If you encounter this error while doing the homework, you can resolve it by restarting the Ollama server using the following command:\n\n```bash\n!nohup ollama serve > nohup.out 2>&1 &\n```\n\nMake sure to rerun the cell containing `ollama serve` if you stop and restart the notebook cell."
  },
  {
    "id": "4d8c1f5b3a",
    "course": "llm-zoomcamp",
    "section": "Module 1: Introduction to LLMs and RAG",
    "question": "How do I count tokens for a non-OpenAI model (Gemini, Mistral, HuggingFace)?",
    "answer": "`tiktoken` only ships tokenizers for OpenAI models. Using `cl100k_base` for other providers gives wrong counts and unreliable cost estimates.\n\nFor other providers, use their native tokenizer:\n\n- Gemini:\n  ```python\n  import google.generativeai as genai\n  model = genai.GenerativeModel('gemini-2.0-flash')\n  print(model.count_tokens(prompt))\n  ```\n- Hugging Face / open-source models:\n  ```python\n  from transformers import AutoTokenizer\n  tok = AutoTokenizer.from_pretrained(\"meta-llama/Llama-3.1-8B\")\n  print(len(tok.encode(prompt)))\n  ```\n- Mistral: use the official `mistral-common` tokenizer package.\n\nDon't use `cl100k_base` as a generic fallback \u2014 token counts will diverge from what the provider actually bills."
  }
]

-> Response received

<IPython.core.display.HTML object>

Assistant:

Use the model name in the command:

ollama run <model-name>

For example:

ollama run llama3

The FAQ only explicitly mentions llama3, so that’s the supported example there. If you want, I can also help with how to call Ollama from Python using a specific model.

In [115]python · cell 49

python

runner.run();

Output

-> Response received

<IPython.core.display.HTML object>

Function call: search({"query":"docker in docker run docker in docker...)

Call

{"query":"docker in docker run docker in docker dind"}

Output

[
  {
    "id": "e8df9f0d12",
    "course": "llm-zoomcamp",
    "section": "Module 6: Best Practices",
    "question": "Docker: When trying to run a streamlit app using docker-compose, I get: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: \"streamlit\": executable file not found in $PATH: unknown. The app runs fine outside of docker-compose",
    "answer": "To resolve this issue:\n\n1. Ensure you have created a `Dockerfile`.\n2. Add `streamlit` to the `docker-compose` configuration.\n3. Run the following command to rebuild and start the application:\n\n   ```bash\n   docker-compose up --build\n   ```"
  },
  {
    "id": "66ccbb7da0",
    "course": "llm-zoomcamp",
    "section": "Module 5: Monitoring",
    "question": "How can I remove all Docker containers, images, and volumes, and builds from the terminal?",
    "answer": "1. Delete all containers (including running ones):\n\n```bash\ndocker rm -f $(docker ps -aq)\n```\n\n2. Remove all images:\n\n```bash\ndocker rmi -f $(docker images -q)\n```\n\n3. Delete all volumes:\n\n```bash\ndocker volume rm $(docker volume ls -q)\n```"
  },
  {
    "id": "42c7af7ed5",
    "course": "llm-zoomcamp",
    "section": "Module 2: Agents",
    "question": "Run MCP Inspector",
    "answer": "To run the MCP Inspector, execute the following command in the terminal:\n\n```bash\nnpx @modelcontextprotocol/inspector\n```"
  },
  {
    "id": "7c4e1a3b58",
    "course": "llm-zoomcamp",
    "section": "Module 2: Agents",
    "question": "Agents: \"RuntimeError: Already running asyncio in this thread\" when calling asyncio.run() from Jupyter",
    "answer": "Jupyter already runs an event loop inside the kernel, so calling `asyncio.run(...)` blows up with:\n\n```\nRuntimeError: Already running asyncio in this thread\n```\n\nIn a notebook cell, just `await` the coroutine directly \u2014 no wrapper needed:\n\n```python\nresult = await main()\n```\n\nFor the FastMCP server in the agents homework, use the async variant:\n\n```python\nawait mcp.run_async()\n```\n\nNote: in the agents homework's \"run the server\" question, you're meant to start the MCP server from a terminal (`python weather_server.py`), not from inside the notebook."
  },
  {
    "id": "92cd22cada",
    "course": "llm-zoomcamp",
    "section": "Module 3: Vector Search",
    "question": "Why was .dot(...) used directly to compute cosine similarity in the lesson, but normalization is emphasized in the homework?",
    "answer": "In the lesson, `.dot(...)` was used under the assumption that the embeddings returned by the model (e.g., `model.encode(...)` from OpenAI) are already normalized to have unit length. In that case, the dot product is mathematically equivalent to cosine similarity.\n\nIn the homework, however, we use classic embeddings like TF-IDF + SVD, which are not normalized by default. This means that the dot product does not represent cosine similarity unless we manually normalize the vectors."
  }
]

<IPython.core.display.HTML object>

Function call: search({"query":"Docker in Docker course FAQ"})

Call

{"query":"Docker in Docker course FAQ"}

Output

[
  {
    "id": "cf93377279",
    "course": "llm-zoomcamp",
    "section": "Module 1: Introduction to LLMs and RAG",
    "question": "ModuleNotFoundError on import docx in parse-faq.ipynb",
    "answer": "The correct package name for `docx` is `python-docx`, not `docx`. Make sure to install the package using:\n\n```bash\npip install python-docx\n```"
  },
  {
    "id": "66ccbb7da0",
    "course": "llm-zoomcamp",
    "section": "Module 5: Monitoring",
    "question": "How can I remove all Docker containers, images, and volumes, and builds from the terminal?",
    "answer": "1. Delete all containers (including running ones):\n\n```bash\ndocker rm -f $(docker ps -aq)\n```\n\n2. Remove all images:\n\n```bash\ndocker rmi -f $(docker images -q)\n```\n\n3. Delete all volumes:\n\n```bash\ndocker volume rm $(docker volume ls -q)\n```"
  },
  {
    "id": "e8df9f0d12",
    "course": "llm-zoomcamp",
    "section": "Module 6: Best Practices",
    "question": "Docker: When trying to run a streamlit app using docker-compose, I get: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: \"streamlit\": executable file not found in $PATH: unknown. The app runs fine outside of docker-compose",
    "answer": "To resolve this issue:\n\n1. Ensure you have created a `Dockerfile`.\n2. Add `streamlit` to the `docker-compose` configuration.\n3. Run the following command to rebuild and start the application:\n\n   ```bash\n   docker-compose up --build\n   ```"
  },
  {
    "id": "c774542f32",
    "course": "llm-zoomcamp",
    "section": "General Course-Related Questions",
    "question": "Server Error (500) When logging in to course homework using GitHub",
    "answer": "Additional error text seen:\n\n```\nThird-Party Login Failure\n\nAn error occurred while attempting to login via your third-party account.\n```\n\nThe current solution is to use Google or Slack to log in and submit homework answers, as the root cause analysis for the GitHub issue is sporadic and doesn\u2019t impact all users."
  },
  {
    "id": "69d122f12e",
    "course": "llm-zoomcamp",
    "section": "General Course-Related Questions",
    "question": "Certificate: Can I follow the course in a self-paced mode and get a certificate?",
    "answer": "No, you can only get a certificate if you finish the course with a \"live\" cohort.\n\nWe don't award certificates for the self-paced mode. The reason is you need to peer-review 3 capstone(s) after submitting your project.\n\nYou can only peer-review projects at the time the course is running; after the form is closed and the peer-review list is compiled."
  }
]

<IPython.core.display.HTML object>

Function call: search({"query":"run docker in docker"})

Call

{"query":"run docker in docker"}

Output

[
  {
    "id": "e8df9f0d12",
    "course": "llm-zoomcamp",
    "section": "Module 6: Best Practices",
    "question": "Docker: When trying to run a streamlit app using docker-compose, I get: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: \"streamlit\": executable file not found in $PATH: unknown. The app runs fine outside of docker-compose",
    "answer": "To resolve this issue:\n\n1. Ensure you have created a `Dockerfile`.\n2. Add `streamlit` to the `docker-compose` configuration.\n3. Run the following command to rebuild and start the application:\n\n   ```bash\n   docker-compose up --build\n   ```"
  },
  {
    "id": "66ccbb7da0",
    "course": "llm-zoomcamp",
    "section": "Module 5: Monitoring",
    "question": "How can I remove all Docker containers, images, and volumes, and builds from the terminal?",
    "answer": "1. Delete all containers (including running ones):\n\n```bash\ndocker rm -f $(docker ps -aq)\n```\n\n2. Remove all images:\n\n```bash\ndocker rmi -f $(docker images -q)\n```\n\n3. Delete all volumes:\n\n```bash\ndocker volume rm $(docker volume ls -q)\n```"
  },
  {
    "id": "42c7af7ed5",
    "course": "llm-zoomcamp",
    "section": "Module 2: Agents",
    "question": "Run MCP Inspector",
    "answer": "To run the MCP Inspector, execute the following command in the terminal:\n\n```bash\nnpx @modelcontextprotocol/inspector\n```"
  },
  {
    "id": "7c4e1a3b58",
    "course": "llm-zoomcamp",
    "section": "Module 2: Agents",
    "question": "Agents: \"RuntimeError: Already running asyncio in this thread\" when calling asyncio.run() from Jupyter",
    "answer": "Jupyter already runs an event loop inside the kernel, so calling `asyncio.run(...)` blows up with:\n\n```\nRuntimeError: Already running asyncio in this thread\n```\n\nIn a notebook cell, just `await` the coroutine directly \u2014 no wrapper needed:\n\n```python\nresult = await main()\n```\n\nFor the FastMCP server in the agents homework, use the async variant:\n\n```python\nawait mcp.run_async()\n```\n\nNote: in the agents homework's \"run the server\" question, you're meant to start the MCP server from a terminal (`python weather_server.py`), not from inside the notebook."
  },
  {
    "id": "92cd22cada",
    "course": "llm-zoomcamp",
    "section": "Module 3: Vector Search",
    "question": "Why was .dot(...) used directly to compute cosine similarity in the lesson, but normalization is emphasized in the homework?",
    "answer": "In the lesson, `.dot(...)` was used under the assumption that the embeddings returned by the model (e.g., `model.encode(...)` from OpenAI) are already normalized to have unit length. In that case, the dot product is mathematically equivalent to cosine similarity.\n\nIn the homework, however, we use classic embeddings like TF-IDF + SVD, which are not normalized by default. This means that the dot product does not represent cosine similarity unless we manually normalize the vectors."
  }
]

-> Response received

<IPython.core.display.HTML object>

Assistant:

I couldn’t find a course FAQ entry specifically about running Docker-in-Docker.

The closest Docker-related FAQ items are about:

rebuilding a Docker Compose app with docker-compose up --build
removing Docker containers/images/volumes with docker rm, docker rmi, and docker volume rm

If you want, I can help look for another course-specific Docker question. Are there other areas you want to explore?

-> Response received

<IPython.core.display.HTML object>

Function call: search({"query":"run ollama in docker ollama docker co...)

Call

{"query":"run ollama in docker ollama docker course FAQ"}

Output

[
  {
    "id": "1d0b969028",
    "course": "llm-zoomcamp",
    "section": "Module 1: Introduction to LLMs and RAG",
    "question": "Ollama: How to install Ollama?",
    "answer": "First, install Ollama by visiting [https://ollama.com/download](https://ollama.com/download) and choosing your operating system:\n\n- **macOS**: Download the `.pkg` and install it.\n- **Windows**: Download the `.msi` and install it.\n- **Linux**: Run the following command in the terminal:\n\n  ```bash\n  curl -fsSL https://ollama.com/install.sh | sh\n  ```\n\nOnce installed, open a terminal and type:\n\n```bash\nollama run llama3\n```\n\nThis command will:\n\n- Download the LLaMA 3 model (~4GB).\n- Start the model locally.\n- Open a chat-like interface where you can type questions.\n\nTo test the Ollama local server, run the following command:\n\n```bash\ncurl http://localhost:11434\n```\n\nYou should receive a response similar to:\n\n```json\n{\"models\": [...]}  \n```\n\nThen, install the Python client with:\n\n```bash\npip install ollama\n```\n\nHere is a minimal Python example:\n\n```python\nimport ollama\n\nresponse = ollama.chat(\n    model='llama3',\n    messages=[{\"role\": \"user\", \"content\": your_prompt}]\n)\n\nprint(response['message']['content'])\n```"
  },
  {
    "id": "cf93377279",
    "course": "llm-zoomcamp",
    "section": "Module 1: Introduction to LLMs and RAG",
    "question": "ModuleNotFoundError on import docx in parse-faq.ipynb",
    "answer": "The correct package name for `docx` is `python-docx`, not `docx`. Make sure to install the package using:\n\n```bash\npip install python-docx\n```"
  },
  {
    "id": "c6fc2d4d11",
    "course": "llm-zoomcamp",
    "section": "Module 2: Agents",
    "question": "Connection refused error on prompting the ollam RAG?",
    "answer": "If you encounter this error while doing the homework, you can resolve it by restarting the Ollama server using the following command:\n\n```bash\n!nohup ollama serve > nohup.out 2>&1 &\n```\n\nMake sure to rerun the cell containing `ollama serve` if you stop and restart the notebook cell."
  },
  {
    "id": "e8df9f0d12",
    "course": "llm-zoomcamp",
    "section": "Module 6: Best Practices",
    "question": "Docker: When trying to run a streamlit app using docker-compose, I get: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: \"streamlit\": executable file not found in $PATH: unknown. The app runs fine outside of docker-compose",
    "answer": "To resolve this issue:\n\n1. Ensure you have created a `Dockerfile`.\n2. Add `streamlit` to the `docker-compose` configuration.\n3. Run the following command to rebuild and start the application:\n\n   ```bash\n   docker-compose up --build\n   ```"
  },
  {
    "id": "66ccbb7da0",
    "course": "llm-zoomcamp",
    "section": "Module 5: Monitoring",
    "question": "How can I remove all Docker containers, images, and volumes, and builds from the terminal?",
    "answer": "1. Delete all containers (including running ones):\n\n```bash\ndocker rm -f $(docker ps -aq)\n```\n\n2. Remove all images:\n\n```bash\ndocker rmi -f $(docker images -q)\n```\n\n3. Delete all volumes:\n\n```bash\ndocker volume rm $(docker volume ls -q)\n```"
  }
]

<IPython.core.display.HTML object>

Function call: search({"query":"ollama docker llm-zoomcamp"})

Call

{"query":"ollama docker llm-zoomcamp"}

Output

[
  {
    "id": "1d0b969028",
    "course": "llm-zoomcamp",
    "section": "Module 1: Introduction to LLMs and RAG",
    "question": "Ollama: How to install Ollama?",
    "answer": "First, install Ollama by visiting [https://ollama.com/download](https://ollama.com/download) and choosing your operating system:\n\n- **macOS**: Download the `.pkg` and install it.\n- **Windows**: Download the `.msi` and install it.\n- **Linux**: Run the following command in the terminal:\n\n  ```bash\n  curl -fsSL https://ollama.com/install.sh | sh\n  ```\n\nOnce installed, open a terminal and type:\n\n```bash\nollama run llama3\n```\n\nThis command will:\n\n- Download the LLaMA 3 model (~4GB).\n- Start the model locally.\n- Open a chat-like interface where you can type questions.\n\nTo test the Ollama local server, run the following command:\n\n```bash\ncurl http://localhost:11434\n```\n\nYou should receive a response similar to:\n\n```json\n{\"models\": [...]}  \n```\n\nThen, install the Python client with:\n\n```bash\npip install ollama\n```\n\nHere is a minimal Python example:\n\n```python\nimport ollama\n\nresponse = ollama.chat(\n    model='llama3',\n    messages=[{\"role\": \"user\", \"content\": your_prompt}]\n)\n\nprint(response['message']['content'])\n```"
  },
  {
    "id": "977bf7786c",
    "course": "llm-zoomcamp",
    "section": "General Course-Related Questions",
    "question": "Course: I have registered for the LLM Zoomcamp. When can I expect to receive the confirmation email?",
    "answer": "You don't need it. You're accepted. You can also just start learning and submitting homework (while the form is open) without registering. It is not checked against any registered list. Registration is just to gauge interest before the start date."
  },
  {
    "id": "a3c7e1b4f9",
    "course": "llm-zoomcamp",
    "section": "Module 4: Evaluation",
    "question": "Evaluation: \"JSONDecodeError: Expecting value\" when generating ground-truth questions with the LLM",
    "answer": "The LLM sometimes wraps the JSON in a markdown code fence or adds prose around it, so `json.loads(response)` fails with:\n\n```\nJSONDecodeError: Expecting value: line 1 column 1 (char 0)\n```\n\nForce JSON output with OpenAI's `response_format`:\n\n```python\nresponse = openai_client.chat.completions.create(\n    model='gpt-4o-mini',\n    messages=[{\"role\": \"user\", \"content\": prompt}],\n    response_format={\"type\": \"json_object\"},\n)\nparsed = json.loads(response.choices[0].message.content)\n```\n\nAlso be explicit in the prompt about the expected shape:\n\n```\nOutput a JSON object with a single key \"questions\" whose value is a list of 5 strings.\nDo not include any extra text, explanation, or formatting.\n```\n\nMost providers have an equivalent (Gemini's `response_mime_type=\"application/json\"`, Groq's `response_format`, etc.)."
  },
  {
    "id": "641aedbbf5",
    "course": "llm-zoomcamp",
    "section": "Module 5: Monitoring",
    "question": "How set Pandas to show entire text content in a column. Useful to view the entire Explanation column content in the LLM-as-judge section of the offline-rag-evaluation notebook",
    "answer": "By default, Pandas truncates text content in a column to 50 characters. To view the entire explanation provided by the judge LLM for a non-relevant answer, use the following instruction:\n\n```python\npd.set_option('display.max_colwidth', None)\n```\n\n- **Option:** `display.max_colwidth`\n- **Type:** `int` or `None`\n- **Description:** Sets the maximum width in characters of a column in the representation of a pandas data structure. When a column overflows, a \"...\" placeholder is used in the output. Setting it to 'None' allows unlimited width.\n- **Default:** 50\n\nRefer to the [official documentation](https://pandas.pydata.org/docs/user_guide/options.html) for more details.\n\n<{IMAGE:image_1}>"
  },
  {
    "id": "c6fc2d4d11",
    "course": "llm-zoomcamp",
    "section": "Module 2: Agents",
    "question": "Connection refused error on prompting the ollam RAG?",
    "answer": "If you encounter this error while doing the homework, you can resolve it by restarting the Ollama server using the following command:\n\n```bash\n!nohup ollama serve > nohup.out 2>&1 &\n```\n\nMake sure to rerun the cell containing `ollama serve` if you stop and restart the notebook cell."
  }
]

<IPython.core.display.HTML object>

Function call: search({"query":"Docker ollama"})

Call

{"query":"Docker ollama"}

Output

[
  {
    "id": "1d0b969028",
    "course": "llm-zoomcamp",
    "section": "Module 1: Introduction to LLMs and RAG",
    "question": "Ollama: How to install Ollama?",
    "answer": "First, install Ollama by visiting [https://ollama.com/download](https://ollama.com/download) and choosing your operating system:\n\n- **macOS**: Download the `.pkg` and install it.\n- **Windows**: Download the `.msi` and install it.\n- **Linux**: Run the following command in the terminal:\n\n  ```bash\n  curl -fsSL https://ollama.com/install.sh | sh\n  ```\n\nOnce installed, open a terminal and type:\n\n```bash\nollama run llama3\n```\n\nThis command will:\n\n- Download the LLaMA 3 model (~4GB).\n- Start the model locally.\n- Open a chat-like interface where you can type questions.\n\nTo test the Ollama local server, run the following command:\n\n```bash\ncurl http://localhost:11434\n```\n\nYou should receive a response similar to:\n\n```json\n{\"models\": [...]}  \n```\n\nThen, install the Python client with:\n\n```bash\npip install ollama\n```\n\nHere is a minimal Python example:\n\n```python\nimport ollama\n\nresponse = ollama.chat(\n    model='llama3',\n    messages=[{\"role\": \"user\", \"content\": your_prompt}]\n)\n\nprint(response['message']['content'])\n```"
  },
  {
    "id": "c6fc2d4d11",
    "course": "llm-zoomcamp",
    "section": "Module 2: Agents",
    "question": "Connection refused error on prompting the ollam RAG?",
    "answer": "If you encounter this error while doing the homework, you can resolve it by restarting the Ollama server using the following command:\n\n```bash\n!nohup ollama serve > nohup.out 2>&1 &\n```\n\nMake sure to rerun the cell containing `ollama serve` if you stop and restart the notebook cell."
  },
  {
    "id": "66ccbb7da0",
    "course": "llm-zoomcamp",
    "section": "Module 5: Monitoring",
    "question": "How can I remove all Docker containers, images, and volumes, and builds from the terminal?",
    "answer": "1. Delete all containers (including running ones):\n\n```bash\ndocker rm -f $(docker ps -aq)\n```\n\n2. Remove all images:\n\n```bash\ndocker rmi -f $(docker images -q)\n```\n\n3. Delete all volumes:\n\n```bash\ndocker volume rm $(docker volume ls -q)\n```"
  },
  {
    "id": "e8df9f0d12",
    "course": "llm-zoomcamp",
    "section": "Module 6: Best Practices",
    "question": "Docker: When trying to run a streamlit app using docker-compose, I get: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: \"streamlit\": executable file not found in $PATH: unknown. The app runs fine outside of docker-compose",
    "answer": "To resolve this issue:\n\n1. Ensure you have created a `Dockerfile`.\n2. Add `streamlit` to the `docker-compose` configuration.\n3. Run the following command to rebuild and start the application:\n\n   ```bash\n   docker-compose up --build\n   ```"
  },
  {
    "id": "2a8f6c4d91",
    "course": "llm-zoomcamp",
    "section": "Module 2: Agents",
    "question": "Agents: which non-OpenAI models support tool / function calling?",
    "answer": "Confirmed working alternatives for tool calling in this course:\n\n- [Groq](https://console.groq.com/docs/tool-use) \u2014 `llama-3.3-70b-versatile`, DeepSeek R1, Llama 4. Free tier, OpenAI-compatible API.\n- [Mistral](https://docs.mistral.ai/capabilities/function_calling/) \u2014 most models. Schema differs slightly from OpenAI.\n- [Google Gemini](https://ai.google.dev/gemini-api/docs/function-calling) \u2014 `gemini-2.5-flash` etc., free tier. Available either through Google's GenAI SDK or via the [OpenAI-compatible endpoint](https://ai.google.dev/gemini-api/docs/openai).\n- [Ollama](https://ollama.com/blog/tool-support) \u2014 `llama3.1` and similar, local. Use `ollama.chat(..., tools=[...])`.\n\nYou'll typically need to adapt the homework's `chat_assistant.py` / `mcp_client.py` slightly when not using OpenAI \u2014 the tool schema and the response shape differ between providers."
  }
]

-> Response received

<IPython.core.display.HTML object>

Assistant:

From the FAQ, the relevant Ollama instructions are:

Install Ollama
- macOS/Windows: download from https://ollama.com/download
- Linux:
```
curl -fsSL https://ollama.com/install.sh | sh
```
Start a model locally:

ollama run llama3

Test the local Ollama server:

curl http://localhost:11434

If you’re using it for the course homework and get a connection refused error, restart the server with:

!nohup ollama serve > nohup.out 2>&1 &

The FAQ also says Ollama supports tool calling, and you can use:

ollama.chat(..., tools=[...])

I couldn’t find a FAQ entry specifically about running Ollama inside Docker, so I can’t give course-specific instructions for that beyond the above.
Is there another area you want to explore?

Chat ended.

In [ ]python · cell 50

python

Chapter 18

agents

Notebookllm-zoomcamp-2026-agents (3.12.1)50 cells

In [1]python · cell 1

python

print(123)

Output

In [2]python · cell 2

python

!wget https://raw.githubusercontent.com/DataTalksClub/llm-zoomcamp/main/01-agentic-rag/code/rag_helper.py
!wget https://raw.githubusercontent.com/DataTalksClub/llm-zoomcamp/main/01-agentic-rag/code/ingest.py

Output

--2026-05-26 09:42:53--  https://raw.githubusercontent.com/DataTalksClub/llm-zoomcamp/main/01-agentic-rag/code/rag_helper.py
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.108.133, 185.199.110.133, 185.199.111.133, ...
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.108.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 2134 (2.1K) [text/plain]
Saving to: ‘rag_helper.py’

rag_helper.py       100%[===================>]   2.08K  --.-KB/s    in 0s      

2026-05-26 09:42:54 (34.7 MB/s) - ‘rag_helper.py’ saved [2134/2134]

--2026-05-26 09:42:54--  https://raw.githubusercontent.com/DataTalksClub/llm-zoomcamp/main/01-agentic-rag/code/ingest.py
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.109.133, 185.199.111.133, 185.199.110.133, ...
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.109.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 738 [text/plain]
Saving to: ‘ingest.py’

ingest.py           100%[===================>]     738  --.-KB/s    in 0s      

2026-05-26 09:42:54 (40.0 MB/s) - ‘ingest.py’ saved [738/738]

In [3]python · cell 3

python

from dotenv import load_dotenv
from openai import OpenAI

load_dotenv()
openai_client = OpenAI()

In [4]python · cell 4

python

from ingest import load_faq_data, build_index

documents = load_faq_data()
index = build_index(documents)

In [5]python · cell 5

python

from rag_helper import RAGBase


instructions = """
You're a course teaching assistant.
Answer the QUESTION based on the CONTEXT from the FAQ database.
Use only the facts from the CONTEXT when answering the QUESTION.
""".strip()

assistant = RAGBase(
    index=index,
    llm_client=openai_client,
    instructions=instructions,
)

In [7]python · cell 6

python

answer = assistant.rag('How do I run Ollama locally?')
print(answer)

Output

To run Ollama locally:

1. Install Ollama from: https://ollama.com/download  
   - macOS: install the `.pkg`
   - Windows: install the `.msi`
   - Linux:
     ```bash
     curl -fsSL https://ollama.com/install.sh | sh
     ```

2. Open a terminal and run:
   ```bash
   ollama run llama3
   ```

This will download the LLaMA 3 model, start it locally, and open a chat-like interface.

3. To test that the local server is running, you can use:
   ```bash
   curl http://localhost:11434
   ```

If needed, you can also install the Python client with:
```bash
pip install ollama
```

In [8]python · cell 7

python

answer = assistant.rag('How do I run Olama locally?')
print(answer)

Output

I don’t have any information in the FAQ context about running Olama locally.

In [11]python · cell 8

python

messages = [
    {'role': 'user', 'content': 'I just discovered the course. Can I join it?'}
]

response = openai_client.responses.create(
    model='gpt-5.4-mini',
    input=messages,
)

response.output_text

Output

'Yes—usually you can, **if enrollment is still open** and the course hasn’t reached capacity.\n\nTo check:\n1. **Look for the course enrollment page** or registration link.\n2. Check whether there’s a **deadline** or **waitlist**.\n3. If it’s through a school, platform, or instructor, **contact them directly** and ask if late enrollment is allowed.\n\nIf you want, I can help you write a short message to ask about joining.'

In [13]python · cell 9

python

def search(query):
    boost_dict = {'question': 3.0, 'section': 0.5}
    filter_dict = {'course': 'llm-zoomcamp'}

    return index.search(
        query,
        num_results=5,
        boost_dict=boost_dict,
        filter_dict=filter_dict
    )

In [17]python · cell 10

python

search_tool = {
    "type": "function",
    'name': 'search',
    'description': 'Search the FAQ database for entries matching the given query.',
    'parameters': {
        "type": "object",
        "properties": {
            'query': {
                "type": "string",
                'description': 'Search query text to look up in the course FAQ.'
            }
        },
        "required": ["query"],
        'additionalProperties': False
    }
}

In [16]python · cell 11

python

response = openai_client.responses.create(
    model='gpt-5.4-mini',
    input=messages,
    tools=[search_tool]
)

In [20]python · cell 12

python

len(response.output)

Output

In [ ]python · cell 13

python

call = response.output[0]

In [22]python · cell 14

python

call

Output

ResponseFunctionToolCall(arguments='{"query":"join course discovered late can I join enroll late join course"}', call_id='call_UpV80e7ba8LoEnDqsjqAfJeC', name='search', type='function_call', id='fc_0f4fa185156188ae006a156dad12e481938c171d34718b305f', namespace=None, status='completed')

In [30]python · cell 15

python

import json

args = json.loads(call.arguments)
args

Output

{'query': 'join course discovered late can I join enroll late join course'}

In [29]python · cell 16

python

call.name

Output

'search'

In [28]python · cell 17

python

results = search(**args)

In [32]python · cell 18

python

result_json = json.dumps(results, indent=2)

In [35]python · cell 19

python

function_call_output = {
    "type": "function_call_output",
    'call_id': call.call_id,
    'output': result_json,
}

In [39]python · cell 20

python

messages.append(call)

In [41]python · cell 21

python

messages.append(function_call_output)

In [42]python · cell 22

python

messages

Output

[{'role': 'user', 'content': 'I just discovered the course. Can I join it?'},
 ResponseFunctionToolCall(arguments='{"query":"join course discovered late can I join enroll late join course"}', call_id='call_UpV80e7ba8LoEnDqsjqAfJeC', name='search', type='function_call', id='fc_0f4fa185156188ae006a156dad12e481938c171d34718b305f', namespace=None, status='completed'),
 {'type': 'function_call_output',
  'call_id': 'call_UpV80e7ba8LoEnDqsjqAfJeC',
  'output': '[\n  {\n    "id": "74eb249bbf",\n    "course": "llm-zoomcamp",\n    "section": "General Course-Related Questions",\n    "question": "I just discovered the course. Can I still join?",\n    "answer": "Yes, but if you want to receive a certificate, you need to submit your project while we\\u2019re still accepting submissions."\n  },\n  {\n    "id": "69d122f12e",\n    "course": "llm-zoomcamp",\n    "section": "General Course-Related Questions",\n    "question": "Certificate: Can I follow the course in a self-paced mode and get a certificate?",\n    "answer": "No, you can only get a certificate if you finish the course with a \\"live\\" cohort.\\n\\nWe don\'t award certificates for the self-paced mode. The reason is you need to peer-review 3 capstone(s) after submitting your project.\\n\\nYou can only peer-review projects at the time the course is running; after the form is closed and the peer-review list is compiled."\n  },\n  {\n    "id": "bd31146b0e",\n    "course": "llm-zoomcamp",\n    "section": "General Course-Related Questions",\n    "question": "When will the course be offered next?",\n    "answer": "Summer 2025."\n  },\n  {\n    "id": "977bf7786c",\n    "course": "llm-zoomcamp",\n    "section": "General Course-Related Questions",\n    "question": "Course: I have registered for the LLM Zoomcamp. When can I expect to receive the confirmation email?",\n    "answer": "You don\'t need it. You\'re accepted. You can also just start learning and submitting homework (while the form is open) without registering. It is not checked against any registered list. Registration is just to gauge interest before the start date."\n  },\n  {\n    "id": "c774542f32",\n    "course": "llm-zoomcamp",\n    "section": "General Course-Related Questions",\n    "question": "Server Error (500) When logging in to course homework using GitHub",\n    "answer": "Additional error text seen:\\n\\n```\\nThird-Party Login Failure\\n\\nAn error occurred while attempting to login via your third-party account.\\n```\\n\\nThe current solution is to use Google or Slack to log in and submit homework answers, as the root cause analysis for the GitHub issue is sporadic and doesn\\u2019t impact all users."\n  }\n]'}]

In [43]python · cell 23

python

response = openai_client.responses.create(
    model='gpt-5.4-mini',
    input=messages,
    tools=[search_tool]
)

In [44]python · cell 24

python

print(response.output_text)

Output

Yes — you can still join the course.

If you want a certificate, though, you need to submit your project while submissions are still open.

In [46]python · cell 25

python

usage = response.usage
usage.input_tokens, usage.output_tokens

Output

(652, 33)

In [48]python · cell 26

python

def calculate_gpt54mini_price(input_tokens, output_tokens):
    # Prices per 1M tokens (example pricing)
    INPUT_PRICE_PER_MILLION = 0.15   # $0.15 / 1M input tokens
    OUTPUT_PRICE_PER_MILLION = 0.60  # $0.60 / 1M output tokens

    input_cost = (input_tokens / 1_000_000) * INPUT_PRICE_PER_MILLION
    output_cost = (output_tokens / 1_000_000) * OUTPUT_PRICE_PER_MILLION

    total_cost = input_cost + output_cost

    return {
        "input_cost": input_cost,
        "output_cost": output_cost,
        "total_cost": total_cost
    }


# Your tokens
result = calculate_gpt54mini_price(652, 33)

print("Total Cost: $", round(result["total_cost"], 8))

Output

Total Cost: $ 0.0001176

In [ ]python · cell 27

python

def make_call(call):
    args = json.loads(call.arguments)

    if call.name == 'search':
        result = search(**args)

    result_json = json.dumps(result, indent=2)

    return {
        "type": "function_call_output",
        'call_id': call.call_id,
        'output': result_json,
    }

In [63]python · cell 28

python

instructions = """
You're a course teaching assistant.
You're given a question from a course student and your task is to answer it.

If you want to look up information, use the search function. 
Use as many keywords from the user question as possible when making first requests.

Make multiple searches.

Try to expand your search by using new keywords
based on the results you get from the search.

At the end, ask if there are other areas that the user wants to explore.
"""

question = 'I just discovered the course. Can I join it?'


messages = [
    {'role': 'developer', 'content': instructions},
    {'role': 'user', 'content': question}
]

In [67]python · cell 29

python

response = openai_client.responses.create(
    model='gpt-5.4-mini',
    input=messages,
    tools=[search_tool]
)

In [ ]python · cell 30

python

messages.extend(response.output)

for item in response.output:
    if item.type == 'function_call':
        print('function_call:', item.name, item.arguments)
        call_output = make_call(item)
        messages.append(call_output)

    elif item.type == 'message':
        print('ASSISTANT:')
        print(item.content[0].text)

Output

Yes, you can still join and follow along.

If you want a certificate, though, you’ll need to submit your project while submissions are still open. Also, certificates are only available for the live cohort, not self-paced.

If you want, I can also help with:
- whether registration is required,
- how homework submission works,
- or when the course runs next.

Is there anything else you’d like to explore?

In [77]python · cell 31

python

messages

Output

[{'role': 'developer',
  'content': "\nYou're a course teaching assistant.\nYou're given a question from a course student and your task is to answer it.\n\nIf you want to look up information, use the search function. \nUse as many keywords from the user question as possible when making first requests.\n\nMake multiple searches.\n\nTry to expand your search by using new keywords\nbased on the results you get from the search.\n\nAt the end, ask if there are other areas that the user wants to explore.\n"},
 {'role': 'user', 'content': 'I just discovered the course. Can I join it?'},
 ResponseFunctionToolCall(arguments='{"query":"join course discovered the course can I join enrollment registration"}', call_id='call_6SVouYKfkyyId3q1pVx8CsJW', name='search', type='function_call', id='fc_008dbefb78b869a2006a1574253d7881928dd1ebd1d2eebe58', namespace=None, status='completed'),
 ResponseFunctionToolCall(arguments='{"query":"course access enrollment late join new student can I join"}', call_id='call_Od8fzAaFVAJ7fKgKfqUSx6rv', name='search', type='function_call', id='fc_008dbefb78b869a2006a1574253d88819284770e1a20aee266', namespace=None, status='completed'),
 {'type': 'function_call_output',
  'call_id': 'call_6SVouYKfkyyId3q1pVx8CsJW',
  'output': '[\n  {\n    "id": "74eb249bbf",\n    "course": "llm-zoomcamp",\n    "section": "General Course-Related Questions",\n    "question": "I just discovered the course. Can I still join?",\n    "answer": "Yes, but if you want to receive a certificate, you need to submit your project while we\\u2019re still accepting submissions."\n  },\n  {\n    "id": "69d122f12e",\n    "course": "llm-zoomcamp",\n    "section": "General Course-Related Questions",\n    "question": "Certificate: Can I follow the course in a self-paced mode and get a certificate?",\n    "answer": "No, you can only get a certificate if you finish the course with a \\"live\\" cohort.\\n\\nWe don\'t award certificates for the self-paced mode. The reason is you need to peer-review 3 capstone(s) after submitting your project.\\n\\nYou can only peer-review projects at the time the course is running; after the form is closed and the peer-review list is compiled."\n  },\n  {\n    "id": "977bf7786c",\n    "course": "llm-zoomcamp",\n    "section": "General Course-Related Questions",\n    "question": "Course: I have registered for the LLM Zoomcamp. When can I expect to receive the confirmation email?",\n    "answer": "You don\'t need it. You\'re accepted. You can also just start learning and submitting homework (while the form is open) without registering. It is not checked against any registered list. Registration is just to gauge interest before the start date."\n  },\n  {\n    "id": "bd31146b0e",\n    "course": "llm-zoomcamp",\n    "section": "General Course-Related Questions",\n    "question": "When will the course be offered next?",\n    "answer": "Summer 2025."\n  },\n  {\n    "id": "c774542f32",\n    "course": "llm-zoomcamp",\n    "section": "General Course-Related Questions",\n    "question": "Server Error (500) When logging in to course homework using GitHub",\n    "answer": "Additional error text seen:\\n\\n```\\nThird-Party Login Failure\\n\\nAn error occurred while attempting to login via your third-party account.\\n```\\n\\nThe current solution is to use Google or Slack to log in and submit homework answers, as the root cause analysis for the GitHub issue is sporadic and doesn\\u2019t impact all users."\n  }\n]'},
 {'type': 'function_call_output',
  'call_id': 'call_Od8fzAaFVAJ7fKgKfqUSx6rv',
  'output': '[\n  {\n    "id": "74eb249bbf",\n    "course": "llm-zoomcamp",\n    "section": "General Course-Related Questions",\n    "question": "I just discovered the course. Can I still join?",\n    "answer": "Yes, but if you want to receive a certificate, you need to submit your project while we\\u2019re still accepting submissions."\n  },\n  {\n    "id": "69d122f12e",\n    "course": "llm-zoomcamp",\n    "section": "General Course-Related Questions",\n    "question": "Certificate: Can I follow the course in a self-paced mode and get a certificate?",\n    "answer": "No, you can only get a certificate if you finish the course with a \\"live\\" cohort.\\n\\nWe don\'t award certificates for the self-paced mode. The reason is you need to peer-review 3 capstone(s) after submitting your project.\\n\\nYou can only peer-review projects at the time the course is running; after the form is closed and the peer-review list is compiled."\n  },\n  {\n    "id": "977bf7786c",\n    "course": "llm-zoomcamp",\n    "section": "General Course-Related Questions",\n    "question": "Course: I have registered for the LLM Zoomcamp. When can I expect to receive the confirmation email?",\n    "answer": "You don\'t need it. You\'re accepted. You can also just start learning and submitting homework (while the form is open) without registering. It is not checked against any registered list. Registration is just to gauge interest before the start date."\n  },\n  {\n    "id": "bd31146b0e",\n    "course": "llm-zoomcamp",\n    "section": "General Course-Related Questions",\n    "question": "When will the course be offered next?",\n    "answer": "Summer 2025."\n  },\n  {\n    "id": "6bceeebf04",\n    "course": "llm-zoomcamp",\n    "section": "Workshop: Open-Source Data Ingestion (dlt)",\n    "question": "How to set up a new dlt project when loading from cloud?",\n    "answer": "Start with the following command on the command line:\\n\\n```bash\\n dlt init filesystem duckdb\\n```\\n\\nMore directions can be found at [dlthub.com](https://dlthub.com/docs/tutorial/filesystem)"\n  }\n]'},
 ResponseOutputMessage(id='msg_008dbefb78b869a2006a15743171708192ae736e385a88251e', content=[ResponseOutputText(annotations=[], text='Yes, you can still join and follow along.\n\nIf you want a certificate, though, you’ll need to submit your project while submissions are still open. Also, certificates are only available for the live cohort, not self-paced.\n\nIf you want, I can also help with:\n- whether registration is required,\n- how homework submission works,\n- or when the course runs next.\n\nIs there anything else you’d like to explore?', type='output_text', logprobs=[])], role='assistant', status='completed', type='message', phase='final_answer')]

In [79]python · cell 32

python

messages = [
    {'role': 'developer', 'content': instructions},
    {'role': 'user', 'content': question}
]

it = 1

while True:
    print(f'iteration #{it}...')
    has_function_calls = False

    response = openai_client.responses.create(
        model='gpt-5.4-mini',
        input=messages,
        tools=[search_tool]
    )

    messages.extend(response.output)

    for item in response.output:
        if item.type == 'function_call':
            print('function_call:', item.name, item.arguments)
            call_output = make_call(item)
            messages.append(call_output)
            has_function_calls = True

        elif item.type == 'message':
            print('ASSISTANT:')
            print(item.content[0].text)
    
    it = it + 1
    if has_function_calls == False:
        break

Output

iteration #1...
function_call: search {"query":"join course discovered course can I join enrollment late joining FAQ"}
function_call: search {"query":"new student can I join course after start FAQ enrollment"}
function_call: search {"query":"course access enrollment deadline can join FAQ"}
iteration #2...
ASSISTANT:
Yes — you can still join the course.

If you want a certificate, make sure to submit your project while submissions are still open. Also, certificates are only available for the live cohort, not self-paced participation.

If you’d like, I can also help with how registration, homework submission, or certificates work.

In [87]python · cell 33

python

def agent_loop(instructions, question, model='gpt-5.4-mini') -> str:
    messages = [
        {'role': 'developer', 'content': instructions},
        {'role': 'user', 'content': question}
    ]

    it = 1

    while True:
        print(f'iteration #{it}...')
        has_function_calls = False

        response = openai_client.responses.create(
            model=model,
            input=messages,
            tools=[search_tool]
        )

        messages.extend(response.output)

        for item in response.output:
            if item.type == 'function_call':
                print('function_call:', item.name, item.arguments)
                call_output = make_call(item)
                messages.append(call_output)
                has_function_calls = True

            elif item.type == 'message':
                print('ASSISTANT:')
                last_answer = item.content[0].text
                print(item.content[0].text)

        it = it + 1
        if has_function_calls == False:
            break
    
    return last_answer

In [88]python · cell 34

python

instructions = """
You're a course teaching assistant.
You're given a question from a course student and your task is to answer it.

If you want to look up information, use the search function. 
Use as many keywords from the user question as possible when making first requests.

Make multiple searches. First perform search, analyze the results 
and then perform more searchers. 

At the end, ask if there are other areas that the user wants to explore.
"""

question = 'I just discovered the course. Can I join it?'

In [90]python · cell 35

python

result = agent_loop(instructions, question)

Output

iteration #1...
function_call: search {"query":"join course discovered course can I join enrollment registration access FAQ"}
iteration #2...
function_call: search {"query":"self-paced join course certificate live cohort project submission accepted while form open FAQ"}
iteration #3...
ASSISTANT:
Yes — you can still join the course.

If you want a certificate, make sure to submit your project while submissions are still being accepted. If you’re only interested in learning, you can start even if you discovered the course later.

If you want, I can also tell you what’s needed to get the certificate or how the course works.

In [91]python · cell 36

python

result

Output

'Yes — you can still join the course.\n\nIf you want a certificate, make sure to submit your project while submissions are still being accepted. If you’re only interested in learning, you can start even if you discovered the course later.\n\nIf you want, I can also tell you what’s needed to get the certificate or how the course works.'

In [93]python · cell 37

python

instructions = """
You're a course teaching assistant.
You're given a question from a course student and your task is to answer it.

If you want to look up information, use the search function. 
Use as many keywords from the user question as possible when making first requests.

Make multiple searches. First perform search, analyze the results 
and then perform more searchers. 

The question has to be about the course or its logistics, offtopic questions 
shouldn't be answered. If the search returns nothing, it's likely an off-topic question.
If you can't answer the question using FAQ, don't do it yourself. Only use the 
facts from the FAQ database.

At the end, ask if there are other areas that the user wants to explore.
"""

question = "what's queen gambit?"

result = agent_loop(instructions, question)

Output

iteration #1...

function_call: search {"query":"queen gambit"}
iteration #2...
function_call: search {"query":"gambit chess opening queen's gambit course FAQ"}
iteration #3...
ASSISTANT:
I couldn’t find a course FAQ entry for “queen gambit,” so it looks like this may be off-topic for the course.

If you meant **“Queen’s Gambit”** in chess, I can’t answer outside the course FAQ. If you meant something course-related and can rephrase it, I’m happy to help.

Is there anything else in the course or its logistics you’d like to explore?

In [95]python · cell 38

python

from toyaikit.llm import OpenAIClient
from toyaikit.tools import Tools
from toyaikit.chat import IPythonChatInterface
from toyaikit.chat.runners import OpenAIResponsesRunner, DisplayingRunnerCallback

In [96]python · cell 39

python

agent_tools = Tools()
agent_tools.add_tool(search, search_tool)

In [99]python · cell 40

python

def search(query: str) -> dict[str, str]:
    """
    Search the FAQ database for entries matching the given query.
    """
    return index.search(
        query,
        num_results=5,
        boost_dict={'question': 3.0, 'section': 0.5},
        filter_dict={'course': 'llm-zoomcamp'}
    )

In [100]python · cell 41

python

agent_tools = Tools()
agent_tools.add_tool(search)

In [101]python · cell 42

python

agent_tools.get_tools()

Output

[{'type': 'function',
  'name': 'search',
  'description': 'Search the FAQ database for entries matching the given query.',
  'parameters': {'type': 'object',
   'properties': {'query': {'type': 'string',
     'description': 'query parameter'}},
   'required': ['query'],
   'additionalProperties': False}}]

In [105]python · cell 43

python

chat_interface = IPythonChatInterface()
callback = DisplayingRunnerCallback(chat_interface)

In [104]python · cell 44

python

runner = OpenAIResponsesRunner(
    tools=agent_tools,
    developer_prompt=instructions,
    chat_interface=chat_interface,
    llm_client=OpenAIClient(model='gpt-5.4-mini')
)

In [107]python · cell 45

python

result = runner.loop(
    prompt='How do I run Olama locally?',
    callback=callback,
)

Output

-> Response received

<IPython.core.display.HTML object>

Function call: search({"query":"Olama locally run install local FAQ"})

Call

{"query":"Olama locally run install local FAQ"}

Output

[
  {
    "id": "cf93377279",
    "course": "llm-zoomcamp",
    "section": "Module 1: Introduction to LLMs and RAG",
    "question": "ModuleNotFoundError on import docx in parse-faq.ipynb",
    "answer": "The correct package name for `docx` is `python-docx`, not `docx`. Make sure to install the package using:\n\n```bash\npip install python-docx\n```"
  },
  {
    "id": "d09e8d4843",
    "course": "llm-zoomcamp",
    "section": "Module 2: Agents",
    "question": "Install MCP Inspector",
    "answer": "1. Ensure Node.js is installed.\n\n2. To install the MCP Inspector, run the following command in your terminal:\n\n   ```bash\n   npm i @modelcontextprotocol/inspector\n   ```"
  },
  {
    "id": "1d0b969028",
    "course": "llm-zoomcamp",
    "section": "Module 1: Introduction to LLMs and RAG",
    "question": "Ollama: How to install Ollama?",
    "answer": "First, install Ollama by visiting [https://ollama.com/download](https://ollama.com/download) and choosing your operating system:\n\n- **macOS**: Download the `.pkg` and install it.\n- **Windows**: Download the `.msi` and install it.\n- **Linux**: Run the following command in the terminal:\n\n  ```bash\n  curl -fsSL https://ollama.com/install.sh | sh\n  ```\n\nOnce installed, open a terminal and type:\n\n```bash\nollama run llama3\n```\n\nThis command will:\n\n- Download the LLaMA 3 model (~4GB).\n- Start the model locally.\n- Open a chat-like interface where you can type questions.\n\nTo test the Ollama local server, run the following command:\n\n```bash\ncurl http://localhost:11434\n```\n\nYou should receive a response similar to:\n\n```json\n{\"models\": [...]}  \n```\n\nThen, install the Python client with:\n\n```bash\npip install ollama\n```\n\nHere is a minimal Python example:\n\n```python\nimport ollama\n\nresponse = ollama.chat(\n    model='llama3',\n    messages=[{\"role\": \"user\", \"content\": your_prompt}]\n)\n\nprint(response['message']['content'])\n```"
  },
  {
    "id": "42c7af7ed5",
    "course": "llm-zoomcamp",
    "section": "Module 2: Agents",
    "question": "Run MCP Inspector",
    "answer": "To run the MCP Inspector, execute the following command in the terminal:\n\n```bash\nnpx @modelcontextprotocol/inspector\n```"
  },
  {
    "id": "15e931476c",
    "course": "llm-zoomcamp",
    "section": "Module 5: Monitoring",
    "question": "In Windows OS: OSError: [WinError 126] The specified module could not be found. Error loading \"C:\\Users\\USER\\AppData\\Local\\Programs\\Python\\Python310\\lib\\site-packages\\torch\\lib\\fbgemm.dll\" or one of its dependencies.",
    "answer": "- **Solution 1**: Install Visual C++ Redistributable.\n\n- **Solution 2**: Install Visual Studio, not Visual Studio Code. \n\n  <{IMAGE:image_1}>\n\n  For more details, please follow this link: [discuss.pytorch.org](https://discuss.pytorch.org/t/failed-to-import-pytorch-fbgemm-dll-or-one-of-its-dependencies-is-missing/201969)"
  }
]

-> Response received

<IPython.core.display.HTML object>

Function call: search({"query":"Ollama run llama3 local server localh...)

Call

{"query":"Ollama run llama3 local server localhost:11434 FAQ"}

Output

[
  {
    "id": "1d0b969028",
    "course": "llm-zoomcamp",
    "section": "Module 1: Introduction to LLMs and RAG",
    "question": "Ollama: How to install Ollama?",
    "answer": "First, install Ollama by visiting [https://ollama.com/download](https://ollama.com/download) and choosing your operating system:\n\n- **macOS**: Download the `.pkg` and install it.\n- **Windows**: Download the `.msi` and install it.\n- **Linux**: Run the following command in the terminal:\n\n  ```bash\n  curl -fsSL https://ollama.com/install.sh | sh\n  ```\n\nOnce installed, open a terminal and type:\n\n```bash\nollama run llama3\n```\n\nThis command will:\n\n- Download the LLaMA 3 model (~4GB).\n- Start the model locally.\n- Open a chat-like interface where you can type questions.\n\nTo test the Ollama local server, run the following command:\n\n```bash\ncurl http://localhost:11434\n```\n\nYou should receive a response similar to:\n\n```json\n{\"models\": [...]}  \n```\n\nThen, install the Python client with:\n\n```bash\npip install ollama\n```\n\nHere is a minimal Python example:\n\n```python\nimport ollama\n\nresponse = ollama.chat(\n    model='llama3',\n    messages=[{\"role\": \"user\", \"content\": your_prompt}]\n)\n\nprint(response['message']['content'])\n```"
  },
  {
    "id": "9c2fd5451c",
    "course": "llm-zoomcamp",
    "section": "Module 2: Agents",
    "question": "Inspect MCP Server",
    "answer": "- Connect to the MCP Server\n\n  <{IMAGE:image_1}>\n\n- The inspector can list tools, templates, resources, and prompts from the MCP Server\n\n  <{IMAGE:image_2}>\n\nReference:\n\n[https://medium.com/@anil.goyal0057/how-to-test-your-mcp-server-using-mcp-inspector-c873c417eec1](https://medium.com/@anil.goyal0057/how-to-test-your-mcp-server-using-mcp-inspector-c873c417eec1)"
  },
  {
    "id": "cf93377279",
    "course": "llm-zoomcamp",
    "section": "Module 1: Introduction to LLMs and RAG",
    "question": "ModuleNotFoundError on import docx in parse-faq.ipynb",
    "answer": "The correct package name for `docx` is `python-docx`, not `docx`. Make sure to install the package using:\n\n```bash\npip install python-docx\n```"
  },
  {
    "id": "42c7af7ed5",
    "course": "llm-zoomcamp",
    "section": "Module 2: Agents",
    "question": "Run MCP Inspector",
    "answer": "To run the MCP Inspector, execute the following command in the terminal:\n\n```bash\nnpx @modelcontextprotocol/inspector\n```"
  },
  {
    "id": "c774542f32",
    "course": "llm-zoomcamp",
    "section": "General Course-Related Questions",
    "question": "Server Error (500) When logging in to course homework using GitHub",
    "answer": "Additional error text seen:\n\n```\nThird-Party Login Failure\n\nAn error occurred while attempting to login via your third-party account.\n```\n\nThe current solution is to use Google or Slack to log in and submit homework answers, as the root cause analysis for the GitHub issue is sporadic and doesn\u2019t impact all users."
  }
]

-> Response received

<IPython.core.display.HTML object>

Assistant:

To run Ollama locally:

Install Ollama from: https://ollama.com/download
- macOS: download and install the .pkg
- Windows: download and install the .msi
- Linux: run
```
curl -fsSL https://ollama.com/install.sh | sh
```
Start a model in your terminal:
```
ollama run llama3
```
This downloads the model, starts it locally, and opens a chat-like interface.
To test that the local server is running:
```
curl http://localhost:11434
```

If you want to use it from Python, install the client:

pip install ollama

and then use:

import ollama

response = ollama.chat(
    model='llama3',
    messages=[{"role": "user", "content": your_prompt}]
)

print(response['message']['content'])

If you'd like, I can also help with how to run a different model or connect Ollama from a notebook.

In [109]python · cell 46

python

result.cost

Output

CostInfo(input_cost=Decimal('0.002517'), output_cost=Decimal('0.0013005'), total_cost=Decimal('0.0038175'))

In [112]python · cell 47

python

result.all_messages

Output

[EasyInputMessage(content="\nYou're a course teaching assistant.\nYou're given a question from a course student and your task is to answer it.\n\nIf you want to look up information, use the search function. \nUse as many keywords from the user question as possible when making first requests.\n\nMake multiple searches. First perform search, analyze the results \nand then perform more searchers. \n\nThe question has to be about the course or its logistics, offtopic questions \nshouldn't be answered. If the search returns nothing, it's likely an off-topic question.\nIf you can't answer the question using FAQ, don't do it yourself. Only use the \nfacts from the FAQ database.\n\nAt the end, ask if there are other areas that the user wants to explore.\n", role='developer', phase=None, type=None),
 EasyInputMessage(content='How do I run Olama locally?', role='user', phase=None, type=None),
 ResponseFunctionToolCall(arguments='{"query":"Olama locally run install local FAQ"}', call_id='call_uQMD3EKYgvzrP1lijpjqVkG1', name='search', type='function_call', id='fc_09b53c21ed00cd32006a157a290eac8195b8a344b9f7e6558a', namespace=None, status='completed'),
 {'type': 'function_call_output',
  'call_id': 'call_uQMD3EKYgvzrP1lijpjqVkG1',
  'output': '[\n  {\n    "id": "cf93377279",\n    "course": "llm-zoomcamp",\n    "section": "Module 1: Introduction to LLMs and RAG",\n    "question": "ModuleNotFoundError on import docx in parse-faq.ipynb",\n    "answer": "The correct package name for `docx` is `python-docx`, not `docx`. Make sure to install the package using:\\n\\n```bash\\npip install python-docx\\n```"\n  },\n  {\n    "id": "d09e8d4843",\n    "course": "llm-zoomcamp",\n    "section": "Module 2: Agents",\n    "question": "Install MCP Inspector",\n    "answer": "1. Ensure Node.js is installed.\\n\\n2. To install the MCP Inspector, run the following command in your terminal:\\n\\n   ```bash\\n   npm i @modelcontextprotocol/inspector\\n   ```"\n  },\n  {\n    "id": "1d0b969028",\n    "course": "llm-zoomcamp",\n    "section": "Module 1: Introduction to LLMs and RAG",\n    "question": "Ollama: How to install Ollama?",\n    "answer": "First, install Ollama by visiting [https://ollama.com/download](https://ollama.com/download) and choosing your operating system:\\n\\n- **macOS**: Download the `.pkg` and install it.\\n- **Windows**: Download the `.msi` and install it.\\n- **Linux**: Run the following command in the terminal:\\n\\n  ```bash\\n  curl -fsSL https://ollama.com/install.sh | sh\\n  ```\\n\\nOnce installed, open a terminal and type:\\n\\n```bash\\nollama run llama3\\n```\\n\\nThis command will:\\n\\n- Download the LLaMA 3 model (~4GB).\\n- Start the model locally.\\n- Open a chat-like interface where you can type questions.\\n\\nTo test the Ollama local server, run the following command:\\n\\n```bash\\ncurl http://localhost:11434\\n```\\n\\nYou should receive a response similar to:\\n\\n```json\\n{\\"models\\": [...]}  \\n```\\n\\nThen, install the Python client with:\\n\\n```bash\\npip install ollama\\n```\\n\\nHere is a minimal Python example:\\n\\n```python\\nimport ollama\\n\\nresponse = ollama.chat(\\n    model=\'llama3\',\\n    messages=[{\\"role\\": \\"user\\", \\"content\\": your_prompt}]\\n)\\n\\nprint(response[\'message\'][\'content\'])\\n```"\n  },\n  {\n    "id": "42c7af7ed5",\n    "course": "llm-zoomcamp",\n    "section": "Module 2: Agents",\n    "question": "Run MCP Inspector",\n    "answer": "To run the MCP Inspector, execute the following command in the terminal:\\n\\n```bash\\nnpx @modelcontextprotocol/inspector\\n```"\n  },\n  {\n    "id": "15e931476c",\n    "course": "llm-zoomcamp",\n    "section": "Module 5: Monitoring",\n    "question": "In Windows OS: OSError: [WinError 126] The specified module could not be found. Error loading \\"C:\\\\Users\\\\USER\\\\AppData\\\\Local\\\\Programs\\\\Python\\\\Python310\\\\lib\\\\site-packages\\\\torch\\\\lib\\\\fbgemm.dll\\" or one of its dependencies.",\n    "answer": "- **Solution 1**: Install Visual C++ Redistributable.\\n\\n- **Solution 2**: Install Visual Studio, not Visual Studio Code. \\n\\n  <{IMAGE:image_1}>\\n\\n  For more details, please follow this link: [discuss.pytorch.org](https://discuss.pytorch.org/t/failed-to-import-pytorch-fbgemm-dll-or-one-of-its-dependencies-is-missing/201969)"\n  }\n]'},
 ResponseFunctionToolCall(arguments='{"query":"Ollama run llama3 local server localhost:11434 FAQ"}', call_id='call_7wA6k4lXW78oODsDNKKKAgK1', name='search', type='function_call', id='fc_09b53c21ed00cd32006a157a2a94108195a68a9890f34d8b32', namespace=None, status='completed'),
 {'type': 'function_call_output',
  'call_id': 'call_7wA6k4lXW78oODsDNKKKAgK1',
  'output': '[\n  {\n    "id": "1d0b969028",\n    "course": "llm-zoomcamp",\n    "section": "Module 1: Introduction to LLMs and RAG",\n    "question": "Ollama: How to install Ollama?",\n    "answer": "First, install Ollama by visiting [https://ollama.com/download](https://ollama.com/download) and choosing your operating system:\\n\\n- **macOS**: Download the `.pkg` and install it.\\n- **Windows**: Download the `.msi` and install it.\\n- **Linux**: Run the following command in the terminal:\\n\\n  ```bash\\n  curl -fsSL https://ollama.com/install.sh | sh\\n  ```\\n\\nOnce installed, open a terminal and type:\\n\\n```bash\\nollama run llama3\\n```\\n\\nThis command will:\\n\\n- Download the LLaMA 3 model (~4GB).\\n- Start the model locally.\\n- Open a chat-like interface where you can type questions.\\n\\nTo test the Ollama local server, run the following command:\\n\\n```bash\\ncurl http://localhost:11434\\n```\\n\\nYou should receive a response similar to:\\n\\n```json\\n{\\"models\\": [...]}  \\n```\\n\\nThen, install the Python client with:\\n\\n```bash\\npip install ollama\\n```\\n\\nHere is a minimal Python example:\\n\\n```python\\nimport ollama\\n\\nresponse = ollama.chat(\\n    model=\'llama3\',\\n    messages=[{\\"role\\": \\"user\\", \\"content\\": your_prompt}]\\n)\\n\\nprint(response[\'message\'][\'content\'])\\n```"\n  },\n  {\n    "id": "9c2fd5451c",\n    "course": "llm-zoomcamp",\n    "section": "Module 2: Agents",\n    "question": "Inspect MCP Server",\n    "answer": "- Connect to the MCP Server\\n\\n  <{IMAGE:image_1}>\\n\\n- The inspector can list tools, templates, resources, and prompts from the MCP Server\\n\\n  <{IMAGE:image_2}>\\n\\nReference:\\n\\n[https://medium.com/@anil.goyal0057/how-to-test-your-mcp-server-using-mcp-inspector-c873c417eec1](https://medium.com/@anil.goyal0057/how-to-test-your-mcp-server-using-mcp-inspector-c873c417eec1)"\n  },\n  {\n    "id": "cf93377279",\n    "course": "llm-zoomcamp",\n    "section": "Module 1: Introduction to LLMs and RAG",\n    "question": "ModuleNotFoundError on import docx in parse-faq.ipynb",\n    "answer": "The correct package name for `docx` is `python-docx`, not `docx`. Make sure to install the package using:\\n\\n```bash\\npip install python-docx\\n```"\n  },\n  {\n    "id": "42c7af7ed5",\n    "course": "llm-zoomcamp",\n    "section": "Module 2: Agents",\n    "question": "Run MCP Inspector",\n    "answer": "To run the MCP Inspector, execute the following command in the terminal:\\n\\n```bash\\nnpx @modelcontextprotocol/inspector\\n```"\n  },\n  {\n    "id": "c774542f32",\n    "course": "llm-zoomcamp",\n    "section": "General Course-Related Questions",\n    "question": "Server Error (500) When logging in to course homework using GitHub",\n    "answer": "Additional error text seen:\\n\\n```\\nThird-Party Login Failure\\n\\nAn error occurred while attempting to login via your third-party account.\\n```\\n\\nThe current solution is to use Google or Slack to log in and submit homework answers, as the root cause analysis for the GitHub issue is sporadic and doesn\\u2019t impact all users."\n  }\n]'},
 ResponseOutputMessage(id='msg_09b53c21ed00cd32006a157a2c226081958a093fd221d7b228', content=[ResponseOutputText(annotations=[], text='To run Ollama locally:\n\n1. Install Ollama from: https://ollama.com/download  \n   - macOS: download and install the `.pkg`\n   - Windows: download and install the `.msi`\n   - Linux: run\n     ```bash\n     curl -fsSL https://ollama.com/install.sh | sh\n     ```\n\n2. Start a model in your terminal:\n   ```bash\n   ollama run llama3\n   ```\n   This downloads the model, starts it locally, and opens a chat-like interface.\n\n3. To test that the local server is running:\n   ```bash\n   curl http://localhost:11434\n   ```\n\nIf you want to use it from Python, install the client:\n```bash\npip install ollama\n```\n\nand then use:\n```python\nimport ollama\n\nresponse = ollama.chat(\n    model=\'llama3\',\n    messages=[{"role": "user", "content": your_prompt}]\n)\n\nprint(response[\'message\'][\'content\'])\n```\n\nIf you\'d like, I can also help with how to run a different model or connect Ollama from a notebook.', type='output_text', logprobs=[])], role='assistant', status='completed', type='message', phase='final_answer')]

In [114]python · cell 48

python

result2 = runner.loop(
    prompt='How do I run a different model?',
    previous_messages=result.all_messages,
    callback=callback,
)

Output

-> Response received

<IPython.core.display.HTML object>

Function call: search({"query":"Ollama different model run llama3 FAQ...)

Call

{"query":"Ollama different model run llama3 FAQ model name ollama run"}

Output

[
  {
    "id": "1d0b969028",
    "course": "llm-zoomcamp",
    "section": "Module 1: Introduction to LLMs and RAG",
    "question": "Ollama: How to install Ollama?",
    "answer": "First, install Ollama by visiting [https://ollama.com/download](https://ollama.com/download) and choosing your operating system:\n\n- **macOS**: Download the `.pkg` and install it.\n- **Windows**: Download the `.msi` and install it.\n- **Linux**: Run the following command in the terminal:\n\n  ```bash\n  curl -fsSL https://ollama.com/install.sh | sh\n  ```\n\nOnce installed, open a terminal and type:\n\n```bash\nollama run llama3\n```\n\nThis command will:\n\n- Download the LLaMA 3 model (~4GB).\n- Start the model locally.\n- Open a chat-like interface where you can type questions.\n\nTo test the Ollama local server, run the following command:\n\n```bash\ncurl http://localhost:11434\n```\n\nYou should receive a response similar to:\n\n```json\n{\"models\": [...]}  \n```\n\nThen, install the Python client with:\n\n```bash\npip install ollama\n```\n\nHere is a minimal Python example:\n\n```python\nimport ollama\n\nresponse = ollama.chat(\n    model='llama3',\n    messages=[{\"role\": \"user\", \"content\": your_prompt}]\n)\n\nprint(response['message']['content'])\n```"
  },
  {
    "id": "42c7af7ed5",
    "course": "llm-zoomcamp",
    "section": "Module 2: Agents",
    "question": "Run MCP Inspector",
    "answer": "To run the MCP Inspector, execute the following command in the terminal:\n\n```bash\nnpx @modelcontextprotocol/inspector\n```"
  },
  {
    "id": "cf93377279",
    "course": "llm-zoomcamp",
    "section": "Module 1: Introduction to LLMs and RAG",
    "question": "ModuleNotFoundError on import docx in parse-faq.ipynb",
    "answer": "The correct package name for `docx` is `python-docx`, not `docx`. Make sure to install the package using:\n\n```bash\npip install python-docx\n```"
  },
  {
    "id": "c6fc2d4d11",
    "course": "llm-zoomcamp",
    "section": "Module 2: Agents",
    "question": "Connection refused error on prompting the ollam RAG?",
    "answer": "If you encounter this error while doing the homework, you can resolve it by restarting the Ollama server using the following command:\n\n```bash\n!nohup ollama serve > nohup.out 2>&1 &\n```\n\nMake sure to rerun the cell containing `ollama serve` if you stop and restart the notebook cell."
  },
  {
    "id": "4d8c1f5b3a",
    "course": "llm-zoomcamp",
    "section": "Module 1: Introduction to LLMs and RAG",
    "question": "How do I count tokens for a non-OpenAI model (Gemini, Mistral, HuggingFace)?",
    "answer": "`tiktoken` only ships tokenizers for OpenAI models. Using `cl100k_base` for other providers gives wrong counts and unreliable cost estimates.\n\nFor other providers, use their native tokenizer:\n\n- Gemini:\n  ```python\n  import google.generativeai as genai\n  model = genai.GenerativeModel('gemini-2.0-flash')\n  print(model.count_tokens(prompt))\n  ```\n- Hugging Face / open-source models:\n  ```python\n  from transformers import AutoTokenizer\n  tok = AutoTokenizer.from_pretrained(\"meta-llama/Llama-3.1-8B\")\n  print(len(tok.encode(prompt)))\n  ```\n- Mistral: use the official `mistral-common` tokenizer package.\n\nDon't use `cl100k_base` as a generic fallback \u2014 token counts will diverge from what the provider actually bills."
  }
]

-> Response received

<IPython.core.display.HTML object>

Assistant:

Use the model name in the command:

ollama run <model-name>

For example:

ollama run llama3

The FAQ only explicitly mentions llama3, so that’s the supported example there. If you want, I can also help with how to call Ollama from Python using a specific model.

In [115]python · cell 49

python

runner.run();

Output

-> Response received

<IPython.core.display.HTML object>

Function call: search({"query":"docker in docker run docker in docker...)

Call

{"query":"docker in docker run docker in docker dind"}

Output

[
  {
    "id": "e8df9f0d12",
    "course": "llm-zoomcamp",
    "section": "Module 6: Best Practices",
    "question": "Docker: When trying to run a streamlit app using docker-compose, I get: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: \"streamlit\": executable file not found in $PATH: unknown. The app runs fine outside of docker-compose",
    "answer": "To resolve this issue:\n\n1. Ensure you have created a `Dockerfile`.\n2. Add `streamlit` to the `docker-compose` configuration.\n3. Run the following command to rebuild and start the application:\n\n   ```bash\n   docker-compose up --build\n   ```"
  },
  {
    "id": "66ccbb7da0",
    "course": "llm-zoomcamp",
    "section": "Module 5: Monitoring",
    "question": "How can I remove all Docker containers, images, and volumes, and builds from the terminal?",
    "answer": "1. Delete all containers (including running ones):\n\n```bash\ndocker rm -f $(docker ps -aq)\n```\n\n2. Remove all images:\n\n```bash\ndocker rmi -f $(docker images -q)\n```\n\n3. Delete all volumes:\n\n```bash\ndocker volume rm $(docker volume ls -q)\n```"
  },
  {
    "id": "42c7af7ed5",
    "course": "llm-zoomcamp",
    "section": "Module 2: Agents",
    "question": "Run MCP Inspector",
    "answer": "To run the MCP Inspector, execute the following command in the terminal:\n\n```bash\nnpx @modelcontextprotocol/inspector\n```"
  },
  {
    "id": "7c4e1a3b58",
    "course": "llm-zoomcamp",
    "section": "Module 2: Agents",
    "question": "Agents: \"RuntimeError: Already running asyncio in this thread\" when calling asyncio.run() from Jupyter",
    "answer": "Jupyter already runs an event loop inside the kernel, so calling `asyncio.run(...)` blows up with:\n\n```\nRuntimeError: Already running asyncio in this thread\n```\n\nIn a notebook cell, just `await` the coroutine directly \u2014 no wrapper needed:\n\n```python\nresult = await main()\n```\n\nFor the FastMCP server in the agents homework, use the async variant:\n\n```python\nawait mcp.run_async()\n```\n\nNote: in the agents homework's \"run the server\" question, you're meant to start the MCP server from a terminal (`python weather_server.py`), not from inside the notebook."
  },
  {
    "id": "92cd22cada",
    "course": "llm-zoomcamp",
    "section": "Module 3: Vector Search",
    "question": "Why was .dot(...) used directly to compute cosine similarity in the lesson, but normalization is emphasized in the homework?",
    "answer": "In the lesson, `.dot(...)` was used under the assumption that the embeddings returned by the model (e.g., `model.encode(...)` from OpenAI) are already normalized to have unit length. In that case, the dot product is mathematically equivalent to cosine similarity.\n\nIn the homework, however, we use classic embeddings like TF-IDF + SVD, which are not normalized by default. This means that the dot product does not represent cosine similarity unless we manually normalize the vectors."
  }
]

<IPython.core.display.HTML object>

Function call: search({"query":"Docker in Docker course FAQ"})

Call

{"query":"Docker in Docker course FAQ"}

Output

[
  {
    "id": "cf93377279",
    "course": "llm-zoomcamp",
    "section": "Module 1: Introduction to LLMs and RAG",
    "question": "ModuleNotFoundError on import docx in parse-faq.ipynb",
    "answer": "The correct package name for `docx` is `python-docx`, not `docx`. Make sure to install the package using:\n\n```bash\npip install python-docx\n```"
  },
  {
    "id": "66ccbb7da0",
    "course": "llm-zoomcamp",
    "section": "Module 5: Monitoring",
    "question": "How can I remove all Docker containers, images, and volumes, and builds from the terminal?",
    "answer": "1. Delete all containers (including running ones):\n\n```bash\ndocker rm -f $(docker ps -aq)\n```\n\n2. Remove all images:\n\n```bash\ndocker rmi -f $(docker images -q)\n```\n\n3. Delete all volumes:\n\n```bash\ndocker volume rm $(docker volume ls -q)\n```"
  },
  {
    "id": "e8df9f0d12",
    "course": "llm-zoomcamp",
    "section": "Module 6: Best Practices",
    "question": "Docker: When trying to run a streamlit app using docker-compose, I get: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: \"streamlit\": executable file not found in $PATH: unknown. The app runs fine outside of docker-compose",
    "answer": "To resolve this issue:\n\n1. Ensure you have created a `Dockerfile`.\n2. Add `streamlit` to the `docker-compose` configuration.\n3. Run the following command to rebuild and start the application:\n\n   ```bash\n   docker-compose up --build\n   ```"
  },
  {
    "id": "c774542f32",
    "course": "llm-zoomcamp",
    "section": "General Course-Related Questions",
    "question": "Server Error (500) When logging in to course homework using GitHub",
    "answer": "Additional error text seen:\n\n```\nThird-Party Login Failure\n\nAn error occurred while attempting to login via your third-party account.\n```\n\nThe current solution is to use Google or Slack to log in and submit homework answers, as the root cause analysis for the GitHub issue is sporadic and doesn\u2019t impact all users."
  },
  {
    "id": "69d122f12e",
    "course": "llm-zoomcamp",
    "section": "General Course-Related Questions",
    "question": "Certificate: Can I follow the course in a self-paced mode and get a certificate?",
    "answer": "No, you can only get a certificate if you finish the course with a \"live\" cohort.\n\nWe don't award certificates for the self-paced mode. The reason is you need to peer-review 3 capstone(s) after submitting your project.\n\nYou can only peer-review projects at the time the course is running; after the form is closed and the peer-review list is compiled."
  }
]

<IPython.core.display.HTML object>

Function call: search({"query":"run docker in docker"})

Call

{"query":"run docker in docker"}

Output

[
  {
    "id": "e8df9f0d12",
    "course": "llm-zoomcamp",
    "section": "Module 6: Best Practices",
    "question": "Docker: When trying to run a streamlit app using docker-compose, I get: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: \"streamlit\": executable file not found in $PATH: unknown. The app runs fine outside of docker-compose",
    "answer": "To resolve this issue:\n\n1. Ensure you have created a `Dockerfile`.\n2. Add `streamlit` to the `docker-compose` configuration.\n3. Run the following command to rebuild and start the application:\n\n   ```bash\n   docker-compose up --build\n   ```"
  },
  {
    "id": "66ccbb7da0",
    "course": "llm-zoomcamp",
    "section": "Module 5: Monitoring",
    "question": "How can I remove all Docker containers, images, and volumes, and builds from the terminal?",
    "answer": "1. Delete all containers (including running ones):\n\n```bash\ndocker rm -f $(docker ps -aq)\n```\n\n2. Remove all images:\n\n```bash\ndocker rmi -f $(docker images -q)\n```\n\n3. Delete all volumes:\n\n```bash\ndocker volume rm $(docker volume ls -q)\n```"
  },
  {
    "id": "42c7af7ed5",
    "course": "llm-zoomcamp",
    "section": "Module 2: Agents",
    "question": "Run MCP Inspector",
    "answer": "To run the MCP Inspector, execute the following command in the terminal:\n\n```bash\nnpx @modelcontextprotocol/inspector\n```"
  },
  {
    "id": "7c4e1a3b58",
    "course": "llm-zoomcamp",
    "section": "Module 2: Agents",
    "question": "Agents: \"RuntimeError: Already running asyncio in this thread\" when calling asyncio.run() from Jupyter",
    "answer": "Jupyter already runs an event loop inside the kernel, so calling `asyncio.run(...)` blows up with:\n\n```\nRuntimeError: Already running asyncio in this thread\n```\n\nIn a notebook cell, just `await` the coroutine directly \u2014 no wrapper needed:\n\n```python\nresult = await main()\n```\n\nFor the FastMCP server in the agents homework, use the async variant:\n\n```python\nawait mcp.run_async()\n```\n\nNote: in the agents homework's \"run the server\" question, you're meant to start the MCP server from a terminal (`python weather_server.py`), not from inside the notebook."
  },
  {
    "id": "92cd22cada",
    "course": "llm-zoomcamp",
    "section": "Module 3: Vector Search",
    "question": "Why was .dot(...) used directly to compute cosine similarity in the lesson, but normalization is emphasized in the homework?",
    "answer": "In the lesson, `.dot(...)` was used under the assumption that the embeddings returned by the model (e.g., `model.encode(...)` from OpenAI) are already normalized to have unit length. In that case, the dot product is mathematically equivalent to cosine similarity.\n\nIn the homework, however, we use classic embeddings like TF-IDF + SVD, which are not normalized by default. This means that the dot product does not represent cosine similarity unless we manually normalize the vectors."
  }
]

-> Response received

<IPython.core.display.HTML object>

Assistant:

I couldn’t find a course FAQ entry specifically about running Docker-in-Docker.

The closest Docker-related FAQ items are about:

rebuilding a Docker Compose app with docker-compose up --build
removing Docker containers/images/volumes with docker rm, docker rmi, and docker volume rm

If you want, I can help look for another course-specific Docker question. Are there other areas you want to explore?

-> Response received

<IPython.core.display.HTML object>

Function call: search({"query":"run ollama in docker ollama docker co...)

Call

{"query":"run ollama in docker ollama docker course FAQ"}

Output

[
  {
    "id": "1d0b969028",
    "course": "llm-zoomcamp",
    "section": "Module 1: Introduction to LLMs and RAG",
    "question": "Ollama: How to install Ollama?",
    "answer": "First, install Ollama by visiting [https://ollama.com/download](https://ollama.com/download) and choosing your operating system:\n\n- **macOS**: Download the `.pkg` and install it.\n- **Windows**: Download the `.msi` and install it.\n- **Linux**: Run the following command in the terminal:\n\n  ```bash\n  curl -fsSL https://ollama.com/install.sh | sh\n  ```\n\nOnce installed, open a terminal and type:\n\n```bash\nollama run llama3\n```\n\nThis command will:\n\n- Download the LLaMA 3 model (~4GB).\n- Start the model locally.\n- Open a chat-like interface where you can type questions.\n\nTo test the Ollama local server, run the following command:\n\n```bash\ncurl http://localhost:11434\n```\n\nYou should receive a response similar to:\n\n```json\n{\"models\": [...]}  \n```\n\nThen, install the Python client with:\n\n```bash\npip install ollama\n```\n\nHere is a minimal Python example:\n\n```python\nimport ollama\n\nresponse = ollama.chat(\n    model='llama3',\n    messages=[{\"role\": \"user\", \"content\": your_prompt}]\n)\n\nprint(response['message']['content'])\n```"
  },
  {
    "id": "cf93377279",
    "course": "llm-zoomcamp",
    "section": "Module 1: Introduction to LLMs and RAG",
    "question": "ModuleNotFoundError on import docx in parse-faq.ipynb",
    "answer": "The correct package name for `docx` is `python-docx`, not `docx`. Make sure to install the package using:\n\n```bash\npip install python-docx\n```"
  },
  {
    "id": "c6fc2d4d11",
    "course": "llm-zoomcamp",
    "section": "Module 2: Agents",
    "question": "Connection refused error on prompting the ollam RAG?",
    "answer": "If you encounter this error while doing the homework, you can resolve it by restarting the Ollama server using the following command:\n\n```bash\n!nohup ollama serve > nohup.out 2>&1 &\n```\n\nMake sure to rerun the cell containing `ollama serve` if you stop and restart the notebook cell."
  },
  {
    "id": "e8df9f0d12",
    "course": "llm-zoomcamp",
    "section": "Module 6: Best Practices",
    "question": "Docker: When trying to run a streamlit app using docker-compose, I get: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: \"streamlit\": executable file not found in $PATH: unknown. The app runs fine outside of docker-compose",
    "answer": "To resolve this issue:\n\n1. Ensure you have created a `Dockerfile`.\n2. Add `streamlit` to the `docker-compose` configuration.\n3. Run the following command to rebuild and start the application:\n\n   ```bash\n   docker-compose up --build\n   ```"
  },
  {
    "id": "66ccbb7da0",
    "course": "llm-zoomcamp",
    "section": "Module 5: Monitoring",
    "question": "How can I remove all Docker containers, images, and volumes, and builds from the terminal?",
    "answer": "1. Delete all containers (including running ones):\n\n```bash\ndocker rm -f $(docker ps -aq)\n```\n\n2. Remove all images:\n\n```bash\ndocker rmi -f $(docker images -q)\n```\n\n3. Delete all volumes:\n\n```bash\ndocker volume rm $(docker volume ls -q)\n```"
  }
]

<IPython.core.display.HTML object>

Function call: search({"query":"ollama docker llm-zoomcamp"})

Call

{"query":"ollama docker llm-zoomcamp"}

Output

[
  {
    "id": "1d0b969028",
    "course": "llm-zoomcamp",
    "section": "Module 1: Introduction to LLMs and RAG",
    "question": "Ollama: How to install Ollama?",
    "answer": "First, install Ollama by visiting [https://ollama.com/download](https://ollama.com/download) and choosing your operating system:\n\n- **macOS**: Download the `.pkg` and install it.\n- **Windows**: Download the `.msi` and install it.\n- **Linux**: Run the following command in the terminal:\n\n  ```bash\n  curl -fsSL https://ollama.com/install.sh | sh\n  ```\n\nOnce installed, open a terminal and type:\n\n```bash\nollama run llama3\n```\n\nThis command will:\n\n- Download the LLaMA 3 model (~4GB).\n- Start the model locally.\n- Open a chat-like interface where you can type questions.\n\nTo test the Ollama local server, run the following command:\n\n```bash\ncurl http://localhost:11434\n```\n\nYou should receive a response similar to:\n\n```json\n{\"models\": [...]}  \n```\n\nThen, install the Python client with:\n\n```bash\npip install ollama\n```\n\nHere is a minimal Python example:\n\n```python\nimport ollama\n\nresponse = ollama.chat(\n    model='llama3',\n    messages=[{\"role\": \"user\", \"content\": your_prompt}]\n)\n\nprint(response['message']['content'])\n```"
  },
  {
    "id": "977bf7786c",
    "course": "llm-zoomcamp",
    "section": "General Course-Related Questions",
    "question": "Course: I have registered for the LLM Zoomcamp. When can I expect to receive the confirmation email?",
    "answer": "You don't need it. You're accepted. You can also just start learning and submitting homework (while the form is open) without registering. It is not checked against any registered list. Registration is just to gauge interest before the start date."
  },
  {
    "id": "a3c7e1b4f9",
    "course": "llm-zoomcamp",
    "section": "Module 4: Evaluation",
    "question": "Evaluation: \"JSONDecodeError: Expecting value\" when generating ground-truth questions with the LLM",
    "answer": "The LLM sometimes wraps the JSON in a markdown code fence or adds prose around it, so `json.loads(response)` fails with:\n\n```\nJSONDecodeError: Expecting value: line 1 column 1 (char 0)\n```\n\nForce JSON output with OpenAI's `response_format`:\n\n```python\nresponse = openai_client.chat.completions.create(\n    model='gpt-4o-mini',\n    messages=[{\"role\": \"user\", \"content\": prompt}],\n    response_format={\"type\": \"json_object\"},\n)\nparsed = json.loads(response.choices[0].message.content)\n```\n\nAlso be explicit in the prompt about the expected shape:\n\n```\nOutput a JSON object with a single key \"questions\" whose value is a list of 5 strings.\nDo not include any extra text, explanation, or formatting.\n```\n\nMost providers have an equivalent (Gemini's `response_mime_type=\"application/json\"`, Groq's `response_format`, etc.)."
  },
  {
    "id": "641aedbbf5",
    "course": "llm-zoomcamp",
    "section": "Module 5: Monitoring",
    "question": "How set Pandas to show entire text content in a column. Useful to view the entire Explanation column content in the LLM-as-judge section of the offline-rag-evaluation notebook",
    "answer": "By default, Pandas truncates text content in a column to 50 characters. To view the entire explanation provided by the judge LLM for a non-relevant answer, use the following instruction:\n\n```python\npd.set_option('display.max_colwidth', None)\n```\n\n- **Option:** `display.max_colwidth`\n- **Type:** `int` or `None`\n- **Description:** Sets the maximum width in characters of a column in the representation of a pandas data structure. When a column overflows, a \"...\" placeholder is used in the output. Setting it to 'None' allows unlimited width.\n- **Default:** 50\n\nRefer to the [official documentation](https://pandas.pydata.org/docs/user_guide/options.html) for more details.\n\n<{IMAGE:image_1}>"
  },
  {
    "id": "c6fc2d4d11",
    "course": "llm-zoomcamp",
    "section": "Module 2: Agents",
    "question": "Connection refused error on prompting the ollam RAG?",
    "answer": "If you encounter this error while doing the homework, you can resolve it by restarting the Ollama server using the following command:\n\n```bash\n!nohup ollama serve > nohup.out 2>&1 &\n```\n\nMake sure to rerun the cell containing `ollama serve` if you stop and restart the notebook cell."
  }
]

<IPython.core.display.HTML object>

Function call: search({"query":"Docker ollama"})

Call

{"query":"Docker ollama"}

Output

[
  {
    "id": "1d0b969028",
    "course": "llm-zoomcamp",
    "section": "Module 1: Introduction to LLMs and RAG",
    "question": "Ollama: How to install Ollama?",
    "answer": "First, install Ollama by visiting [https://ollama.com/download](https://ollama.com/download) and choosing your operating system:\n\n- **macOS**: Download the `.pkg` and install it.\n- **Windows**: Download the `.msi` and install it.\n- **Linux**: Run the following command in the terminal:\n\n  ```bash\n  curl -fsSL https://ollama.com/install.sh | sh\n  ```\n\nOnce installed, open a terminal and type:\n\n```bash\nollama run llama3\n```\n\nThis command will:\n\n- Download the LLaMA 3 model (~4GB).\n- Start the model locally.\n- Open a chat-like interface where you can type questions.\n\nTo test the Ollama local server, run the following command:\n\n```bash\ncurl http://localhost:11434\n```\n\nYou should receive a response similar to:\n\n```json\n{\"models\": [...]}  \n```\n\nThen, install the Python client with:\n\n```bash\npip install ollama\n```\n\nHere is a minimal Python example:\n\n```python\nimport ollama\n\nresponse = ollama.chat(\n    model='llama3',\n    messages=[{\"role\": \"user\", \"content\": your_prompt}]\n)\n\nprint(response['message']['content'])\n```"
  },
  {
    "id": "c6fc2d4d11",
    "course": "llm-zoomcamp",
    "section": "Module 2: Agents",
    "question": "Connection refused error on prompting the ollam RAG?",
    "answer": "If you encounter this error while doing the homework, you can resolve it by restarting the Ollama server using the following command:\n\n```bash\n!nohup ollama serve > nohup.out 2>&1 &\n```\n\nMake sure to rerun the cell containing `ollama serve` if you stop and restart the notebook cell."
  },
  {
    "id": "66ccbb7da0",
    "course": "llm-zoomcamp",
    "section": "Module 5: Monitoring",
    "question": "How can I remove all Docker containers, images, and volumes, and builds from the terminal?",
    "answer": "1. Delete all containers (including running ones):\n\n```bash\ndocker rm -f $(docker ps -aq)\n```\n\n2. Remove all images:\n\n```bash\ndocker rmi -f $(docker images -q)\n```\n\n3. Delete all volumes:\n\n```bash\ndocker volume rm $(docker volume ls -q)\n```"
  },
  {
    "id": "e8df9f0d12",
    "course": "llm-zoomcamp",
    "section": "Module 6: Best Practices",
    "question": "Docker: When trying to run a streamlit app using docker-compose, I get: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: \"streamlit\": executable file not found in $PATH: unknown. The app runs fine outside of docker-compose",
    "answer": "To resolve this issue:\n\n1. Ensure you have created a `Dockerfile`.\n2. Add `streamlit` to the `docker-compose` configuration.\n3. Run the following command to rebuild and start the application:\n\n   ```bash\n   docker-compose up --build\n   ```"
  },
  {
    "id": "2a8f6c4d91",
    "course": "llm-zoomcamp",
    "section": "Module 2: Agents",
    "question": "Agents: which non-OpenAI models support tool / function calling?",
    "answer": "Confirmed working alternatives for tool calling in this course:\n\n- [Groq](https://console.groq.com/docs/tool-use) \u2014 `llama-3.3-70b-versatile`, DeepSeek R1, Llama 4. Free tier, OpenAI-compatible API.\n- [Mistral](https://docs.mistral.ai/capabilities/function_calling/) \u2014 most models. Schema differs slightly from OpenAI.\n- [Google Gemini](https://ai.google.dev/gemini-api/docs/function-calling) \u2014 `gemini-2.5-flash` etc., free tier. Available either through Google's GenAI SDK or via the [OpenAI-compatible endpoint](https://ai.google.dev/gemini-api/docs/openai).\n- [Ollama](https://ollama.com/blog/tool-support) \u2014 `llama3.1` and similar, local. Use `ollama.chat(..., tools=[...])`.\n\nYou'll typically need to adapt the homework's `chat_assistant.py` / `mcp_client.py` slightly when not using OpenAI \u2014 the tool schema and the response shape differ between providers."
  }
]

-> Response received

<IPython.core.display.HTML object>

Assistant:

From the FAQ, the relevant Ollama instructions are:

Install Ollama
- macOS/Windows: download from https://ollama.com/download
- Linux:
```
curl -fsSL https://ollama.com/install.sh | sh
```
Start a model locally:

ollama run llama3

Test the local Ollama server:

curl http://localhost:11434

If you’re using it for the course homework and get a connection refused error, restart the server with:

!nohup ollama serve > nohup.out 2>&1 &

The FAQ also says Ollama supports tool calling, and you can use:

ollama.chat(..., tools=[...])

I couldn’t find a FAQ entry specifically about running Ollama inside Docker, so I can’t give course-specific instructions for that beyond the above.
Is there another area you want to explore?

Chat ended.

In [ ]python · cell 50

python