Chapter 61
Evaluation: agent-trace (Siddhant-K-code/agent-trace)
Evaluation: agent-trace (Siddhant-K-code/agent-trace)
Date: 2026-03-16 Source: https://github.com/Siddhant-K-code/agent-trace Type: GitHub repository (Python tool) Evaluator: Claude (eval-resource skill)
Summary
agent-trace (pip package: agent-strace) is a Python tool (zero dependencies, stdlib only) that captures every tool call, user prompt, and assistant response in Claude Code via hooks, then lets you replay sessions in the terminal or export as OpenTelemetry spans. Created 2026-03-15. 7 stars at time of evaluation (now 77 as of 2026-07-28).
The "strace for AI agents" framing is apt: it solves the "my agent modified 47 files and I have no idea why" problem by giving you a time-stamped, replayable record of every decision point.
Key Points
- Claude Code hooks: Setup via
agent-strace setup. Registers PreToolUse, PostToolUse, PostToolUseFailure, UserPromptSubmit, Stop, SessionStart, SessionEnd in.claude/settings.json - Session replay:
agent-strace replayshows full session with timestamps, durations, tool inputs, errors — the missing layer between JSONL and understanding - MCP proxy: Wraps any MCP server (stdio or HTTP/SSE). Works with Cursor, Windsurf, any MCP client
- OpenTelemetry export: OTLP output → Datadog, Honeycomb, New Relic, Splunk
- Python decorator API:
@trace_tool,@trace_llm_call,log_decision()for custom agents - Secret redaction:
--redactflag strips OpenAI, GitHub, AWS, Anthropic, Slack, JWTs, Bearer tokens, connection strings
Relevance Score: 2/5
Pertinent but too immature for immediate integration.
The session replay angle is real and not covered by existing tools in the guide. But claude-code-otel already handles the OTel export use case, and the manual jq queries at guide/ops/observability.md:519-550 cover most of the audit use case. The unique differentiator — interactive replay — needs production validation before being recommended to readers.
Comparison vs Current Guide Coverage
| Aspect | agent-trace | Guide coverage |
|---|---|---|
| Manual JSONL audit (jq) | ✅ Abstracted as CLI | ✅ observability.md:520 |
| Session replay (visual) | ✅ Unique differentiator | ❌ Not covered |
| OpenTelemetry export | ✅ OTLP | ✅ claude-code-otel already in table |
| Hook setup automation | ✅ agent-strace setup | ✅ Documented manually |
| MCP proxy (Cursor/Windsurf) | ✅ stdio + HTTP/SSE | ❌ Not covered |
| Python decorator API | ✅ Custom agents | ❌ Not covered |
| Maturity | ❌ 1 day old, 7 stars | ✅ Table tools have 100-10K stars |
Challenge Notes (technical-writer review)
Score should be 2/5, not 3/5. Reasons:
claude-code-otelalready exports to Datadog/Honeycomb. The OTel angle is not additive.- The jq queries at observability.md:519-550 cover most of the audit use case already. The "replay niche" is thinner than it appears.
- ICM (1 star) was put on watch list. Agent-trace at 7 stars deserves the same treatment.
Missing aspects not in initial analysis:
- MCP proxy = MITM risk: Routing all MCP traffic through an unaudited HTTP/SSE proxy is a security surface. The guide has a full hardening section — adding this to the monitoring table without flagging would be inconsistent.
- Secret redaction unverified: Base64-encoded tokens, multi-line .env values, AWS temporary credentials — edge cases not tested. Could create false confidence.
- Python decorator API vs MLflow SDK: MLflow has versioning + experiment tracking + LLM-as-judge. Agent-trace has lower friction. Real trade-off not mentioned.
On placement: If integrated, not in the External Monitoring Tools table (that's monitoring, not debugging). Better as a footnote in the JSONL section (~observability.md:565) as "a higher-level wrapper for session replay."
Risk of NOT integrating: Near zero. The jq queries + claude-code-otel cover the primary use cases. Real risk runs the other direction: adding a 1-day-old tool that goes unmaintained = dead link in a table readers use for tooling decisions.
Fact-Check
| Claim | Verified | Source |
|---|---|---|
| Zero dependencies, Python stdlib only | ✅ | pyproject.toml + README |
| Created 2026-03-15 | ✅ | GitHub API: created_at: 2026-03-15T08:09:45Z |
| MIT licensed | ✅ | GitHub API: license: MIT License |
| Captures all CC hook events | ✅ | README hooks JSON: all 7 event types |
| Export to Datadog, Honeycomb, Splunk | ✅ | README: export --to otlp (OTLP compatible) |
| 7 stars at evaluation | ✅ | GitHub API 2026-03-16 |
No hallucinations detected. All stats confirmed against source.
Decision
Action: Watch list Integration trigger: 100+ stars AND at least one practitioner write-up showing real production use.
If triggered: Add as footnote in observability.md ~line 565 (JSONL section), not in the External Monitoring Tools table. Frame as "higher-level wrapper for session replay/debug" distinct from the monitoring tools.
Why watch list and not reject: Session replay is a real gap. Zero-deps Python is a genuine adoption differentiator. The engineering quality looks solid (automated setup, secret redaction, HTTP/SSE proxy). Just needs time to prove reliability on real sessions.