-
Notifications
You must be signed in to change notification settings - Fork 191
Improve OTel GenAI semantic convention compliance for LLM instrumentations #1593
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
jimilp7
wants to merge
8
commits into
pydantic:update-llm-instrumentation-semconvs
Choose a base branch
from
introspection-dev:update-llm-instrumentation-semconvs
base: update-llm-instrumentation-semconvs
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
abec7c9 to
265e892
Compare
…tions This commit enhances OpenAI and Anthropic instrumentations to better comply with OpenTelemetry GenAI semantic conventions (https://opentelemetry.io/docs/specs/semconv/gen-ai/). Changes: - Set SpanKind=CLIENT for all GenAI API calls (was INTERNAL) - Use OTel-compliant span names in "{operation} {model}" format (e.g., "chat gpt-4o-mini") - Add request parameter attributes: gen_ai.request.max_tokens, temperature, top_p, top_k, stop_sequences, seed, frequency_penalty, presence_penalty - Add gen_ai.tool.definitions when tools are provided in requests - Add server.address and server.port attributes extracted from client base_url - Remove deprecated gen_ai.system attribute from OpenAI instrumentation - Add finish_reasons mapping for OpenAI Responses API (status → finish_reason) Files modified: - main.py: Added _span_kind parameter support to Logfire.span() - llm_provider.py: Set SpanKind.CLIENT, OTel span naming, server attributes - openai.py: Request params extraction, tool definitions, remove deprecated attr - anthropic.py: Request params extraction, tool definitions - semconv.py: Added new semantic convention constants 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]>
265e892 to
eec1209
Compare
Add LangChain/LangGraph OTel GenAI semantic convention support - Transform LangSmith spans to add OTel GenAI attributes - Add gen_ai.input.messages and gen_ai.output.messages in OTel schema - Extract gen_ai.system_instructions from system messages - Add gen_ai.response.finish_reasons - Add logfire.instrument_langchain() to capture tool definitions - Patches BaseCallbackManager to inject callback handler - Captures gen_ai.tool.definitions (not available via LangSmith OTEL) - Uses _start()/_end() pattern to avoid async context issues
5121d6a to
9307923
Compare
47a633e to
015babe
Compare
54a2096 to
2f34d18
Compare
e41de92 to
d35ecc4
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR improves OTel GenAI Semantic Convention compliance across LLM instrumentations.
What's New
1. OpenAI & Anthropic Instrumentation Improvements
Enhanced existing instrumentations to better comply with OTel GenAI Semantic Conventions:
INTERNALCLIENT"Chat Completion with {model!r}""chat gpt-4o-mini"gen_ai.system"openai"(deprecated)gen_ai.provider.name"openai"/"anthropic"server.address"api.openai.com"gen_ai.request.*max_tokens,temperature,top_p, etc.gen_ai.tool.definitions2. LangChain/LangGraph Instrumentation (New)
Adds
logfire.instrument_langchain()to capture tool definitions from LangChain agents.Why: LangSmith's OTEL integration does NOT include tool definitions - they're sent via their proprietary API. This instrumentation patches LangChain's callback system to capture them.
Usage:
Captures:
Related