Skip to content

Conversation

@jimilp7
Copy link

@jimilp7 jimilp7 commented Dec 24, 2025

Summary

This PR improves OTel GenAI Semantic Convention compliance across LLM instrumentations.

What's New

1. OpenAI & Anthropic Instrumentation Improvements

Enhanced existing instrumentations to better comply with OTel GenAI Semantic Conventions:

Attribute Before After
SpanKind INTERNAL CLIENT
SpanName "Chat Completion with {model!r}" "chat gpt-4o-mini"
gen_ai.system "openai" (deprecated) Removed
gen_ai.provider.name Missing "openai" / "anthropic"
server.address Missing "api.openai.com"
gen_ai.request.* Missing max_tokens, temperature, top_p, etc.
gen_ai.tool.definitions Missing Tool schemas when provided

2. LangChain/LangGraph Instrumentation (New)

Adds logfire.instrument_langchain() to capture tool definitions from LangChain agents.

Why: LangSmith's OTEL integration does NOT include tool definitions - they're sent via their proprietary API. This instrumentation patches LangChain's callback system to capture them.

Usage:

import logfire
from langgraph.prebuilt import create_react_agent

logfire.configure()
logfire.instrument_langchain()

agent = create_react_agent(model, tools)
agent.invoke({"messages": [...]})

Captures:

  • gen_ai.tool.definitions - Tool schemas (name, description, parameters)
  • gen_ai.input.messages / gen_ai.output.messages - OTel format
  • gen_ai.system_instructions - Extracted from system messages
  • Token usage and model information

Related

@jimilp7 jimilp7 changed the title Update llm instrumentation semconvs Improve OTel GenAI semantic convention compliance for LLM instrumentations Dec 24, 2025
@jimilp7 jimilp7 changed the base branch from main to update-llm-instrumentation-semconvs December 24, 2025 00:32
@jimilp7 jimilp7 marked this pull request as draft December 24, 2025 00:33
@jimilp7 jimilp7 force-pushed the update-llm-instrumentation-semconvs branch 2 times, most recently from abec7c9 to 265e892 Compare December 24, 2025 01:08
…tions

This commit enhances OpenAI and Anthropic instrumentations to better comply
with OpenTelemetry GenAI semantic conventions (https://opentelemetry.io/docs/specs/semconv/gen-ai/).

Changes:
- Set SpanKind=CLIENT for all GenAI API calls (was INTERNAL)
- Use OTel-compliant span names in "{operation} {model}" format (e.g., "chat gpt-4o-mini")
- Add request parameter attributes: gen_ai.request.max_tokens, temperature, top_p, top_k,
  stop_sequences, seed, frequency_penalty, presence_penalty
- Add gen_ai.tool.definitions when tools are provided in requests
- Add server.address and server.port attributes extracted from client base_url
- Remove deprecated gen_ai.system attribute from OpenAI instrumentation
- Add finish_reasons mapping for OpenAI Responses API (status → finish_reason)

Files modified:
- main.py: Added _span_kind parameter support to Logfire.span()
- llm_provider.py: Set SpanKind.CLIENT, OTel span naming, server attributes
- openai.py: Request params extraction, tool definitions, remove deprecated attr
- anthropic.py: Request params extraction, tool definitions
- semconv.py: Added new semantic convention constants

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <[email protected]>
@jimilp7 jimilp7 force-pushed the update-llm-instrumentation-semconvs branch from 265e892 to eec1209 Compare December 24, 2025 23:37
Add LangChain/LangGraph OTel GenAI semantic convention support
  - Transform LangSmith spans to add OTel GenAI attributes
  - Add gen_ai.input.messages and gen_ai.output.messages in OTel schema
  - Extract gen_ai.system_instructions from system messages
  - Add gen_ai.response.finish_reasons
  - Add logfire.instrument_langchain() to capture tool definitions
  - Patches BaseCallbackManager to inject callback handler
  - Captures gen_ai.tool.definitions (not available via LangSmith OTEL)
  - Uses _start()/_end() pattern to avoid async context issues
@jimilp7 jimilp7 force-pushed the update-llm-instrumentation-semconvs branch from 5121d6a to 9307923 Compare December 29, 2025 22:27
@jimilp7 jimilp7 force-pushed the update-llm-instrumentation-semconvs branch from 47a633e to 015babe Compare December 30, 2025 02:18
@jimilp7 jimilp7 force-pushed the update-llm-instrumentation-semconvs branch from 54a2096 to 2f34d18 Compare December 30, 2025 02:48
@jimilp7 jimilp7 force-pushed the update-llm-instrumentation-semconvs branch from e41de92 to d35ecc4 Compare December 30, 2025 02:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant