Workshop Notebooks

Interactive Jupyter Notebooks for Hands-On Edge AI Learning

Progressive, self-paced tutorials that build from basic chat completions to advanced multi-agent systems using Microsoft Foundry Local and Small Language Models.

📖 Introduction

Welcome to the EdgeAI for Beginners Workshop Notebooks collection. These interactive Jupyter notebooks provide a hands-on learning experience where you'll write, execute, and experiment with Edge AI code in real-time.

Why Jupyter Notebooks?

Unlike traditional tutorials, these notebooks offer:

• Interactive Learning: Run code cells and see immediate results
• Experimentation: Modify parameters and observe changes in real-time
• Documentation: Inline explanations and markdown cells guide you through concepts
• Reproducibility: Complete working examples you can reference and reuse
• Visualization: View performance metrics, embeddings, and results inline

What Makes These Notebooks Special?

Each notebook is designed following production-ready best practices:

✅ Comprehensive Error Handling - Graceful degradation and informative error messages
✅ Type Hints & Documentation - Clear function signatures and docstrings
✅ Performance Monitoring - Token usage tracking and latency measurements
✅ Modular Design - Reusable patterns you can adapt to your projects
✅ Progressive Complexity - Builds on previous sessions systematically

🎯 Learning Objectives

Core Skills You'll Develop

By working through these notebooks, you will master:

1. Local AI Service Management

   • Configure and manage Microsoft Foundry Local services
   • Select and load appropriate models for your hardware
   • Monitor resource usage and optimize performance
   • Handle service discovery and health checking

2. AI Application Development

   • Implement OpenAI-compatible chat completions locally
   • Build streaming interfaces for better user experience
   • Design effective prompts for Small Language Models
   • Integrate local models into applications

3. Retrieval Augmented Generation (RAG)

   • Create semantic search with vector embeddings
   • Ground LLM responses in domain-specific documents
   • Evaluate RAG quality with RAGAS metrics
   • Scale from prototype to production

4. Performance Optimization

   • Benchmark multiple models systematically
   • Measure latency, throughput, and first-token time
   • Compare Small Language Models vs Large Language Models
   • Select optimal models based on performance/quality trade-offs

5. Multi-Agent Orchestration

   • Design specialized agents for different tasks
   • Implement agent memory and context management
   • Coordinate multiple agents in complex workflows
   • Build coordinator patterns for agent collaboration

6. Intelligent Model Routing

   • Implement intent detection and pattern matching
   • Route queries to appropriate models automatically
   • Build multi-step pipelines (plan → execute → refine)
   • Design scalable model-as-tools architectures

🎓 Learning Outcomes

What You'll Build

Competency Progression

Session 01 ────► Session 02 ────► Session 03 ────► Session 04 ────► Session 05-06
   Basic            RAG             Benchmarking      Comparison      Multi-Agent
   Chat             Systems         & Performance     & Optimization  Orchestration
   
Foundation    │   Intermediate   │   Advanced       │   Expert
──────────────┴──────────────────┴──────────────────┴──────────────►

📅 Workshop Schedule

🚀 Half-Day Workshop (3.5 hours)

Perfect for: Team training sessions, hackathons, conference workshops

Output: Attendees leave with 6 working Edge AI applications and production-ready code patterns.

🎓 Full-Day Workshop (6 hours)

Perfect for: In-depth training, bootcamps, university courses

Output: Deep understanding of Edge AI patterns plus 2 custom projects.

📚 Self-Paced Learning (2 weeks)

Perfect for: Individual learners, online courses, self-study

Week 1: Foundations (6 hours)

Week 2: Advanced (5 hours)

Output: Mastery of Edge AI patterns plus portfolio project.

📔 Notebook Descriptions

📘 Session 01: Chat Bootstrap

File: session01_chat_bootstrap.ipynb
Duration: 20-30 minutes
Prerequisites: None
Difficulty: ⭐ Beginner

What You'll Learn:

• Install and configure Foundry Local Python SDK
• Use FoundryLocalManager for automatic service discovery
• Implement basic chat completions with OpenAI-compatible API
• Build streaming responses for better user experience
• Handle errors and service unavailability gracefully

Key Concepts: Service management, chat completions, streaming, error handling

You'll Build: Interactive chat application with streaming support

📗 Session 02: RAG Pipeline

File: session02_rag_pipeline.ipynb
Duration: 30-45 minutes
Prerequisites: Session 01
Difficulty: ⭐⭐ Intermediate

What You'll Learn:

• Implement Retrieval Augmented Generation (RAG) pattern
• Create vector embeddings with sentence-transformers
• Build semantic search with cosine similarity
• Ground LLM responses in domain documents
• Handle optional dependencies with import guards

Key Concepts: RAG architecture, embeddings, semantic search, vector similarity

You'll Build: Document-grounded question-answering system

📗 Session 02: RAG Evaluation with RAGAS

File: session02_rag_eval_ragas.ipynb
Duration: 30-45 minutes
Prerequisites: Session 02 RAG Pipeline
Difficulty: ⭐⭐ Intermediate

What You'll Learn:

• Evaluate RAG quality with industry-standard metrics
• Measure context relevance, answer relevance, faithfulness
• Use RAGAS framework for systematic evaluation
• Identify and fix RAG quality issues
• Build evaluation datasets for your domain

Key Concepts: RAG evaluation, RAGAS metrics, quality measurement, systematic testing

You'll Build: RAG quality evaluation framework

📙 Session 03: Benchmark OSS Models

File: session03_benchmark_oss_models.ipynb
Duration: 30-45 minutes
Prerequisites: Session 01
Difficulty: ⭐⭐ Intermediate

What You'll Learn:

• Systematically benchmark multiple models
• Measure latency, throughput, first-token time
• Implement graceful degradation for model failures
• Compare performance across model families
• Visualize and analyze benchmark results

Key Concepts: Performance benchmarking, latency measurement, model comparison, statistical analysis

You'll Build: Multi-model benchmarking suite

📙 Session 04: Model Comparison (SLM vs LLM)

File: session04_model_compare.ipynb
Duration: 30-45 minutes
Prerequisites: Sessions 01, 03
Difficulty: ⭐⭐⭐ Advanced

What You'll Learn:

• Compare Small Language Models vs Large Language Models
• Analyze performance vs quality trade-offs
• Measure edge-suitability metrics
• Select optimal models for deployment constraints
• Document decision criteria for model selection

Key Concepts: Model selection, trade-off analysis, optimization strategies, deployment planning

You'll Build: SLM vs LLM comparison framework

📕 Session 05: Multi-Agent Orchestrator

File: session05_agents_orchestrator.ipynb
Duration: 45-60 minutes
Prerequisites: Sessions 01-02
Difficulty: ⭐⭐⭐ Advanced

What You'll Learn:

• Design specialized agents for different tasks
• Implement agent memory and context management
• Build coordinator patterns for agent collaboration
• Handle agent communication and handoffs
• Monitor multi-agent system performance

Key Concepts: Agent architecture, coordinator patterns, memory management, agent orchestration

You'll Build: Multi-agent system with coordinator and specialists

📕 Session 06: Model Router

File: session06_models_router.ipynb
Duration: 30-45 minutes
Prerequisites: Sessions 01, 03
Difficulty: ⭐⭐⭐ Advanced

What You'll Learn:

• Implement intent detection and pattern matching
• Build keyword-based model routing
• Route queries to appropriate models automatically
• Configure multi-model registries
• Monitor routing decisions and performance

Key Concepts: Intent detection, model routing, pattern matching, intelligent selection

You'll Build: Intelligent model routing system

📕 Session 06: Multi-Step Pipeline

File: session06_models_pipeline.ipynb
Duration: 30-45 minutes
Prerequisites: Sessions 01, 06 Router
Difficulty: ⭐⭐⭐ Advanced

What You'll Learn:

• Build multi-step AI pipelines (plan → execute → refine)
• Integrate router for intelligent model selection
• Implement pipeline error handling and recovery
• Monitor pipeline performance and stages
• Design scalable model-as-tools architectures

Key Concepts: Pipeline architecture, multi-stage processing, error recovery, scalability patterns

You'll Build: Multi-step intelligent pipeline with routing

🚀 Getting Started

Prerequisites

System Requirements:

• OS: Windows 10/11, macOS 11+, or Linux (Ubuntu 20.04+)
• RAM: 8GB minimum, 16GB+ recommended
• Storage: 10GB+ free space for models
• Hardware: CPU with AVX2; GPU (CUDA, Qualcomm NPU) optional

Software Requirements:

• Python 3.8+ with pip
• Jupyter Notebook or VS Code with Jupyter extension
• Microsoft Foundry Local installed and configured
• Git (for cloning repository)

Installation Steps

1. Install Foundry Local

Windows:

winget install Microsoft.FoundryLocal

macOS:

brew tap microsoft/foundrylocal
brew install foundrylocal

Verify Installation:

foundry --version

2. Set Up Python Environment

# Navigate to Workshop directory
cd Workshop

# Create virtual environment
python -m venv .venv

# Activate virtual environment
# Windows
.venv\Scripts\activate
# macOS/Linux
source .venv/bin/activate

# Install dependencies
pip install -r requirements.txt

3. Start Foundry Local

# Load a model (auto-downloads if needed)
foundry model run phi-4-mini

# Verify service is running
foundry service status

4. Launch Jupyter

# Start Jupyter Notebook
jupyter notebook notebooks/

# Or use VS Code with Jupyter extension
code notebooks/

Quick Verification

Run this in a Python cell to verify setup:

from foundry_local import FoundryLocalManager
import openai

# Initialize manager (auto-discovers service)
manager = FoundryLocalManager("phi-4-mini")

# Configure OpenAI client
client = openai.OpenAI(
    base_url=manager.endpoint,
    api_key=manager.api_key
)

# Test chat completion
response = client.chat.completions.create(
    model=manager.get_model_info("phi-4-mini").id,
    messages=[{"role": "user", "content": "Hello!"}]
)

print(response.choices[0].message.content)

Expected Output: A greeting response from the local model.

📝 Workshop Best Practices

For Instructors

Before the Workshop:

• ✅ Send installation instructions 1 week in advance
• ✅ Test all notebooks on target hardware
• ✅ Prepare troubleshooting guide for common issues
• ✅ Have backup models ready (phi-3.5-mini if phi-4-mini fails)
• ✅ Set up shared chat channel for questions

During the Workshop:

• ✅ Start with quick environment check (5 minutes)
• ✅ Share troubleshooting resources immediately
• ✅ Encourage experimentation and modifications
• ✅ Use breaks strategically (after every 2 sessions)
• ✅ Have TAs available for 1-on-1 help

After the Workshop:

• ✅ Share complete working notebooks and solutions
• ✅ Provide links to additional resources
• ✅ Create feedback survey for improvement
• ✅ Offer office hours for follow-up questions

For Learners

Maximize Your Learning:

• ✅ Complete setup before workshop starts
• ✅ Run every code cell yourself (don't just read)
• ✅ Experiment with parameters and prompts
• ✅ Take notes on insights and gotchas
• ✅ Ask questions when stuck (others likely have same question)

Common Pitfalls to Avoid:

• ❌ Skipping cell execution order (run sequentially)
• ❌ Not reading error messages carefully
• ❌ Rushing through without understanding
• ❌ Ignoring markdown explanations
• ❌ Not saving your modified notebooks

Debugging Tips:

1. Service Not Running: Check foundry service status
2. Import Errors: Verify virtual environment is activated
3. Model Not Found: Run foundry model ls to list loaded models
4. Slow Performance: Check RAM usage, close other applications
5. Unexpected Results: Restart kernel and run all cells from top

🔗 Additional Resources

Workshop Materials

• Workshop Main Guide - Overview, learning objectives, career outcomes
• Python Samples - Corresponding Python scripts for each session
• Session Guides - Detailed markdown guides (Session01-06)
• Scripts - Validation and testing utilities
• Troubleshooting - Common issues and solutions
• Quick Start - Fast-track getting started guide

Documentation

• Foundry Local Docs - Official Microsoft documentation
• OpenAI Python SDK - OpenAI SDK reference
• Sentence Transformers - Embedding models documentation
• RAGAS Framework - RAG evaluation metrics

Community

• GitHub Discussions - Ask questions, share projects
• Azure AI Foundry Discord - Real-time community support
• Stack Overflow - Technical Q&A

🎯 Learning Path Recommendations

Beginner Track (Start Here)

1. Session 01 - Chat Bootstrap
2. Session 02 - RAG Pipeline
3. Session 03 - Benchmark Models

Time: ~2 hours | Focus: Foundational patterns

Intermediate Track

1. Complete Beginner Track
2. Session 02 - RAG Evaluation
3. Session 04 - Model Comparison

Time: ~4 hours | Focus: Quality and optimization

Advanced Track (Full Workshop)

1. Complete Intermediate Track
2. Session 05 - Multi-Agent Orchestrator
3. Session 06 - Model Router
4. Session 06 - Multi-Step Pipeline

Time: ~6 hours | Focus: Production patterns

Custom Project Track

1. Complete Beginner Track (Sessions 01-03)
2. Choose ONE advanced session based on your goal:
   • Building RAG app? → Session 02 Evaluation
   • Optimizing performance? → Session 04 Comparison
   • Complex workflows? → Session 05 Orchestrator
   • Scalable architecture? → Session 06 Router + Pipeline

Time: ~3 hours | Focus: Project-specific skills

📊 Success Metrics

Track your progress with these milestones:

• Setup Complete - Foundry Local running, all dependencies installed
• First Chat - Session 01 completed, streaming chat working
• RAG Built - Session 02 completed, document QA system functional
• Models Benchmarked - Session 03 completed, performance data collected
• Trade-offs Analyzed - Session 04 completed, model selection criteria documented
• Agents Orchestrated - Session 05 completed, multi-agent system working
• Routing Implemented - Session 06 completed, intelligent model selection functional
• Custom Project - Applied workshop patterns to your own use case

🤝 Contributing

Found an issue or have a suggestion? We welcome contributions!

• Report Issues: GitHub Issues
• Suggest Improvements: GitHub Discussions
• Submit PRs: Follow Contributing Guidelines

📄 License

This workshop is part of the EdgeAI for Beginners repository and is licensed under the MIT License.

Ready to build production-ready Edge AI applications?
Start with Session 01: Chat Bootstrap →

Last Updated: October 8, 2025 | Workshop Version: 2.0