-
Notifications
You must be signed in to change notification settings - Fork 461
feat(telemetry): Add tool definitions to traces via semconv opt-in #1113
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the contribution.
After the discussion within the team, I believe we should:
-
Move the attributes to invoke_agent span > gen_ai.tool.definitions
(semantic conventions link). -
Move the flag (include_tool_definitions) into StrandsTelemetry class instead of in the Agent class as this is specifically for the trace.
Thanks for the feedback, will change it right away @poshinchen . |
|
Oh no, I messed up something by force pushing since I was having some rebase problems locally and now it closed the PR automatically. Sorry, could you reopen it or should I create a new PR @poshinchen . Nvm i was able to reopen it. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi, sorry
After a careful read from the otel documentation. We are expecting customers to set multiple values in OTEL_SEMCONV_STABILITY_OPT_IN env var
- As an example:
OTEL_SEMCONV_STABILITY_OPT_IN="gen_ai_latest_experimental,gen_ai_tool_definitions"allows latest semantic conventions, and opt-in gen_ai.tool.definitions.
That being said, could you do the following:
- In tracer.py, modify the parsing logic for use_latest_genai_conventions and include_tool_definitions (new).
- pass all_tools_config to
_start_agent_trace_spanand move tool_details = [...] into that method. - I wonder if it's possible to pass them as array?
- From the description:
It’s expected to be an array of objects where each object represents a tool definition. In case a serialized string is available to the instrumentation, the instrumentation SHOULD do the best effort to deserialize it to an array. When recorded on spans, it MAY be recorded as a JSON string if structured format is not supported and SHOULD be recorded in structured form otherwise.
No problem , will change it according your feedback. Thanks. Also, regarding your question about passing the data as an array: I looked into it, and you're right that the spec prefers a structured format. However, the OpenTelemetry Python SDK's set_attribute function doesn't have guaranteed support for complex nested objects like a list of dictionaries. What are your thoughts on this, open to being wrong about this. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you update the description and the commit too?
I have updated the commit and the description. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Change the tests please
Its changed now, thanks for the help. |
Description
This PR implements a feature to include tool definitions in agent traces (invoke_agent span), following the OpenTelemetry semantic conventions for GenAI.
The feature is enabled when gen_ai_tool_definitions is included in the
OTEL_SEMCONV_STABILITY_OPT_INenvironment variable (viaexport OTEL_SEMCONV_STABILITY_OPT_IN="gen_ai_tool_definitions,...")Key Changes
OTEL_SEMCONV_STABILITY_OPT_INenvironment variable and validate ifgen_ai_tool_definitionsexists to include all available tools' configs.Related Issues
Closes #1083
Documentation PR
N/A
Type of Change
New feature
Testing
How have you tested the change? Verify that the changes do not break functionality or introduce warnings in consuming repositories: agents-docs, agents-tools, agents-cli
hatch run prepareChecklist
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.