Chapter 27

basic usage

NotebookPython 3 (ipykernel)24 cells

Basic Ollama Usage Examples

This shows the simplest ways to replace OpenAI/other API calls with Ollama. Perfect for understanding the core concepts before moving to agents.

Direct API call
Ollama API parameters
Basic LangChain Integration
Conversation

In [ ]python · cell 2

python

import requests
import time

Lets check if Ollama is runninig:

In [ ]python · cell 4

python

def check_ollama_running():
    """Check if Ollama is available."""
    try:
        response = requests.get("http://localhost:11434/api/tags", timeout=5)
        return response.status_code == 200
    except:
        return False

if not check_ollama_running():
    print("❌ Ollama not running!")
    print("Start it with: ollama serve")
    print("And make sure you have a model: ollama pull llama3.1:8b")
else:
    print("✅ Ollama is running\n")
    print("🚀 Basic Ollama Usage Examples")
    print("=" * 40)

Direct API

system vs user call:

✅ role: "system" Purpose: Sets initial behavior, tone, or constraints for the model. Example Use: Define the assistant's personality, scope, or instructions before the conversation starts. The model reads it once and uses it to shape all following responses. Not shown to the user in most applications.

✅ role: "user" Purpose: Represents the actual question or input from the user. Triggers the model to respond. Can be followed by assistant messages to continue the back-and-forth dialogue.

In [ ]python · cell 7

python

def example_1_direct_api(sys_promt, usr_promt):
    # This replaces OpenAI API calls
    # response = requests.post("http://localhost:11434/api/chat", json={
    #     "model": "llama3.1:8b",
    #     "messages": [
    #         {"role": "system", "content": "You are a helpful assistant."},
    #         {"role": "user", "content": "What is Python?"}
    #     ]
    # })
    response = requests.post("http://localhost:11434/api/chat", json={
        "model": "llama3.1:8b",
        "messages": [{"role": "system", "content": sys_promt},
                     {"role": "user", "content":  usr_promt}
                     ],
        "stream": False  # Ensure it's not streaming
    })
    if response.status_code == 200:
        result = response.json()
        print(f"Response: {result["message"]["content"]}")
    else:
        print(f"Error: {response.status_code}")

In [ ]python · cell 8

python

print("Example 1: Direct API Call")
print("-" * 30)
sys_promt = "You are a helpful assistant."
usr_promt = "What is Python?"
example_1_direct_api(sys_promt,usr_promt)

Ollama API Parameters

Ollama provides extensive customization options. Here are the most commonly used parameters:

Parameter	Type	Default	Description
`model`	string	-	Required. Model identifier (e.g., `"llama3.1:8b"`)
`messages`	array	-	Chat conversation for `/api/chat`
`stream`	boolean	true	Stream response as it's generated
`temperature`	float	0.8	Randomness (0.0 = deterministic, 2.0 = very random)
`top_p`	float	0.9	Nucleus sampling (0.1 = top 10% tokens only)
`num_predict`	int	128	Maximum tokens to generate (-1 = unlimited)
`repeat_penalty`	float	1.1	Penalty for repeating tokens
`system`	string	-	System message to guide behavior
`stop`	array	-	Stop generation at these strings

In [ ]python · cell 10

python

def example_2_with_parameters(temperature, num_predict, usr_promt):
   
    response = requests.post("http://localhost:11434/api/chat", json={
        "model": "llama3.1:8b",
        "messages": [{"role": "user", "content": usr_promt}],
        "temperature": temperature,  # Higher temperature for creativity
        "num_predict": num_predict,   # Limit response length
        "stream": False  # Ensure it's not streaming
    })
    
    if response.status_code == 200:
        result = response.json()["message"]["content"]
        print(f"Creative response: {result}")
    else:
        print(f"Error: {response.status_code}")

In [ ]python · cell 11

python

print("\nExample 2: With Parameters")
print("-" * 30)
usr_promt = "Write a creative story about a robot."
temperature = 0.8
num_predict = 100
example_2_with_parameters(temperature, num_predict, usr_promt)

LangChain Integration

Feature	ChatOllama	Direct API
`Abstraction level`	High	Low
`Ease of use`	Easier	Manual formatting required
`Customization (headers, etc.)`	Limited	Full
`Dependency`	Requires LangChain	Only requests
`Best for`	RAG pipelines, fast prototyping	Custom tools, low-level control

In [ ]python · cell 13

python

def example_3_langchain_basic(usr_promt):    
    try:
        from langchain_community.chat_models import ChatOllama
        
        # Replace ChatOpenAI with ChatOllama
        llm = ChatOllama(model="llama3.1:8b", temperature=0.1)
        
        response = llm.invoke(usr_promt)
        print(f"LangChain response: {response.content}")
        
    except ImportError:
        print("LangChain not installed. Run: pip install langchain langchain-community")
    except Exception as e:
        print(f"Error: {e}")

In [ ]python · cell 14

python

print("\nExample 3: LangChain Integration")
print("-" * 30)
usr_promt = "Explain machine learning in one sentence."
example_3_langchain_basic(usr_promt)

Maintaining Conversation Context

In a conversation with a language model, messages are exchanged between different roles. The "assistant" role represents the model’s response to a user input. When we add {"role": "assistant", "content": assistant_msg} to the message history, we’re storing the model’s last reply. This helps maintain context in multi-turn conversations, allowing the model to remember what it said before and respond accordingly.

In [ ]python · cell 17

python

def example_4_conversation(messages):   
    response = requests.post("http://localhost:11434/api/chat", json={
        "model": "llama3.1:8b",
        "messages": messages,
        "stream": False
    })
    
    if response.status_code == 200:
        assistant_msg = response.json()["message"]["content"]
        return assistant_msg

Start conversation:

In [ ]python · cell 19

python

print("\nExample 4: Conversation")
print("-" * 30)
messages = [{"role": "system", "content": "You are a helpful coding assistant."}]
messages.append({"role": "user", "content": "How do I read a file in Python?"})
assistant_msg = example_4_conversation(messages)
print(f"Assistant:\n{assistant_msg}")

Add assistant:

In [ ]python · cell 21

python

messages.append({"role": "assistant", "content": assistant_msg})

Add follow-up question:

In [ ]python · cell 23

python

messages.append({"role": "user", "content": "What about error handling?"})
answer = example_4_conversation(messages)
print(f"\n\nFollow-up:\n{answer}")

✅ All examples completed!

💡 Key Points:

• Replace OpenAI URLs with http://localhost:11434

• No API keys needed!

• ChatOllama replaces ChatOpenAI in LangChain

def check_ollama_running(): """Check if Ollama is available.""" try: response = requests.get("http://localhost:11434/api/tags", timeout=5) return response.status_code == 200 except: return False if not check_ollama_running(): print("❌ Ollama not running!") print("Start it with: ollama serve") print("And make sure you have a model: ollama pull llama3.1:8b") else: print("✅ Ollama is running\n") print("🚀 Basic Ollama Usage Examples") print("=" * 40)

system vs user call:

✅ role: "user" Purpose: Represents the actual question or input from the user. Triggers the model to respond. Can be followed by assistant messages to continue the back-and-forth dialogue.

def example_1_direct_api(sys_promt, usr_promt): # This replaces OpenAI API calls # response = requests.post("http://localhost:11434/api/chat", json={ # "model": "llama3.1:8b", # "messages": [ # {"role": "system", "content": "You are a helpful assistant."}, # {"role": "user", "content": "What is Python?"} # ] # }) response = requests.post("http://localhost:11434/api/chat", json={ "model": "llama3.1:8b", "messages": [{"role": "system", "content": sys_promt}, {"role": "user", "content": usr_promt} ], "stream": False # Ensure it's not streaming }) if response.status_code == 200: result = response.json() print(f"Response: {result["message"]["content"]}") else: print(f"Error: {response.status_code}")

Ollama API Parameters

Ollama provides extensive customization options. Here are the most commonly used parameters:

Parameter	Type	Default	Description
`model`	string	-	Required. Model identifier (e.g., `"llama3.1:8b"`)
`messages`	array	-	Chat conversation for `/api/chat`
`stream`	boolean	true	Stream response as it's generated
`temperature`	float	0.8	Randomness (0.0 = deterministic, 2.0 = very random)
`top_p`	float	0.9	Nucleus sampling (0.1 = top 10% tokens only)
`num_predict`	int	128	Maximum tokens to generate (-1 = unlimited)
`repeat_penalty`	float	1.1	Penalty for repeating tokens
`system`	string	-	System message to guide behavior
`stop`	array	-	Stop generation at these strings

def example_2_with_parameters(temperature, num_predict, usr_promt): response = requests.post("http://localhost:11434/api/chat", json={ "model": "llama3.1:8b", "messages": [{"role": "user", "content": usr_promt}], "temperature": temperature, # Higher temperature for creativity "num_predict": num_predict, # Limit response length "stream": False # Ensure it's not streaming }) if response.status_code == 200: result = response.json()["message"]["content"] print(f"Creative response: {result}") else: print(f"Error: {response.status_code}")

print("\nExample 2: With Parameters") print("-" * 30) usr_promt = "Write a creative story about a robot." temperature = 0.8 num_predict = 100 example_2_with_parameters(temperature, num_predict, usr_promt)

Feature

ChatOllama

Direct API

Abstraction level

High

Low

Ease of use

Easier

Manual formatting required

Customization (headers, etc.)

Limited

Full

Dependency

Requires LangChain

Only requests

Best for

RAG pipelines, fast prototyping

Custom tools, low-level control

def example_3_langchain_basic(usr_promt): try: from langchain_community.chat_models import ChatOllama # Replace ChatOpenAI with ChatOllama llm = ChatOllama(model="llama3.1:8b", temperature=0.1) response = llm.invoke(usr_promt) print(f"LangChain response: {response.content}") except ImportError: print("LangChain not installed. Run: pip install langchain langchain-community") except Exception as e: print(f"Error: {e}")

def example_4_conversation(messages): response = requests.post("http://localhost:11434/api/chat", json={ "model": "llama3.1:8b", "messages": messages, "stream": False }) if response.status_code == 200: assistant_msg = response.json()["message"]["content"] return assistant_msg

print("\nExample 4: Conversation") print("-" * 30) messages = [{"role": "system", "content": "You are a helpful coding assistant."}] messages.append({"role": "user", "content": "How do I read a file in Python?"}) assistant_msg = example_4_conversation(messages) print(f"Assistant:\n{assistant_msg}")