Workshop Scripts

This directory contains automation and support scripts used to maintain quality and consistency across the Workshop materials.

Contents

1. Markdown CLI Pattern Linter

lint_markdown_cli.py scans all non-translation .md files for disallowed Foundry Local CLI patterns inside fenced code blocks (...). Informational prose can still mention deprecated commands for historical context.

Deprecated Patterns (Blocked Inside Code Fences)

The linter blocks deprecated CLI patterns. Use modern alternatives instead.

Required Replacements

Exit Codes

Running Locally

From the repository root (recommended):

Windows (PowerShell):

python Workshop\scripts\lint_markdown_cli.py --verbose

macOS / Linux:

python Workshop/scripts/lint_markdown_cli.py --verbose

Pre-Commit Hook (Optional)

echo "python Workshop/scripts/lint_markdown_cli.py" > .git/hooks/pre-commit
chmod +x .git/hooks/pre-commit

This blocks commits that introduce deprecated patterns.

CI Integration

A GitHub Action workflow (.github/workflows/markdown-cli-lint.yml) runs the linter on every push and pull request to main and Reactor branches. Failing jobs must be fixed before merging.

Adding New Deprecated Patterns

1. Open lint_markdown_cli.py.
2. Append a tuple (regex, suggestion) to the DEPRECATED list. Use a raw string and include \b word boundaries where appropriate.
3. Run the linter locally to verify detection.
4. Commit and push; CI will enforce the new rule.

Example addition:

DEPRECATED.append((r"\\bfoundry\\s+experimental\\s+foo\\b", "Remove experimental foo usage"))

Allowing Explanatory Mentions

Because only fenced code blocks are enforced, you may describe deprecated commands in narrative text safely. If you must show them inside a fence for contrast, add a fenced block without triple backticks (e.g., indent or quote) or rewrite to pseudo form.

Skipping Specific Files (Advanced)

If needed, wrap legacy examples in a separate file outside the repo or rename with a different extension while drafting. Intentional skips for translated copies are automatic (paths containing translations).

Troubleshooting

Rationale

Keeping documentation aligned with current stable CLI surface prevents workshop friction, avoids learner confusion, and centralizes performance measurement through maintained Python scripts instead of drifting CLI subcommands.

Maintained as part of the workshop quality toolchain. For enhancements (e.g., auto-fixing suggestions or HTML report generation), open an issue or submit a PR.

2. Sample Validation Script

validate_samples.py validates all Python sample files for syntax, imports, and best practices compliance.

Usage

# Validate all samples
python scripts/validate_samples.py

# Validate specific session
python scripts/validate_samples.py --session 01

# Verbose mode (includes best practice warnings)
python scripts/validate_samples.py --verbose

# Summary only
python scripts/validate_samples.py --summary

What it checks

• ✅ Python syntax validity
• ✅ Required imports present
• ✅ Error handling implementation (verbose mode)
• ✅ Type hints usage (verbose mode)
• ✅ Function docstrings (verbose mode)
• ✅ SDK reference links (verbose mode)

Environment Variables

• SKIP_IMPORT_CHECK=1 - Skip import validation
• SKIP_SYNTAX_CHECK=1 - Skip syntax validation

Exit Codes

• 0 - All samples passed validation
• 1 - One or more samples failed

3. Sample Test Runner

test_samples.py runs smoke tests on all samples to verify they execute without errors.

Usage

# Test all samples
python scripts/test_samples.py

# Test specific session
python scripts/test_samples.py --session 01

# Quick mode (shorter timeouts)
python scripts/test_samples.py --quick

# Verbose mode (show output)
python scripts/test_samples.py --verbose

Prerequisites

• Foundry Local service running: foundry service start
• Models loaded: foundry model run phi-4-mini
• Dependencies installed: pip install -r requirements.txt

What it tests

• ✅ Sample can execute without runtime errors
• ✅ Required output is generated
• ✅ Proper error handling on failure
• ✅ Performance (execution time)

Environment Variables

• FOUNDRY_LOCAL_ALIAS=phi-4-mini - Model to use for testing
• TEST_TIMEOUT=30 - Timeout per sample in seconds

Expected Failures

Some tests may fail if optional dependencies are not installed (e.g., ragas, sentence-transformers). Install with:

pip install sentence-transformers ragas datasets

Exit Codes

• 0 - All tests passed
• 1 - One or more tests failed

4. Benchmark Markdown Exporter

Script: export_benchmark_markdown.py

Generates a reproducible performance table for a set of models.

Usage

python Workshop\scripts\export_benchmark_markdown.py --models "qwen2.5-0.5b" --prompt "Explain retrieval augmented generation briefly." --rounds 3 --output benchmark_report.md

Outputs

Options

Environment variable BENCH_STREAM=1 adds first token latency measurement.

Notes

• Reuses workshop_utils for consistent model bootstrap & caching.
• If run from a different working directory, script attempts path fallbacks to locate workshop_utils.
• For GPU comparison: run once, enable acceleration via CLI config, re-run and diff the JSON.