Skip to content

Conversation

@Ratish1
Copy link
Contributor

@Ratish1 Ratish1 commented Oct 30, 2025

Description

This PR implements a feature to include tool definitions in agent traces (invoke_agent span), following the OpenTelemetry semantic conventions for GenAI.

The feature is enabled when gen_ai_tool_definitions is included in the OTEL_SEMCONV_STABILITY_OPT_IN environment variable (via export OTEL_SEMCONV_STABILITY_OPT_IN="gen_ai_tool_definitions,...")

Key Changes

  • The tracer parses the OTEL_SEMCONV_STABILITY_OPT_IN environment variable and validate if gen_ai_tool_definitions exists to include all available tools' configs.
  • The gen_ai.agent.tools attribute (containing just the tool names) has been preserved for backward compatibility.
  • Unit tests have been updated to test the environment variable configuration.

Related Issues

Closes #1083

Documentation PR

N/A

Type of Change

New feature

Testing

How have you tested the change? Verify that the changes do not break functionality or introduce warnings in consuming repositories: agents-docs, agents-tools, agents-cli

  • I ran hatch run prepare
  • I ran unit tests:
  • test_agent_does_not_include_tools_in_trace_by_default -> passed
  • test_agent_includes_tools_in_trace_when_enabled -> passed

Checklist

  • I have read the CONTRIBUTING document
  • I have added any necessary tests that prove my fix is effective or my feature works
  • I have updated the documentation accordingly
  • I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
  • My changes generate no new warnings
  • Any dependent changes have been merged and published

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@Ratish1 Ratish1 changed the title feat(agent): Add opt-in flag to include tool specs in trace feat(agent): Add opt-in flag to include tool specs in traces for evaluation Oct 30, 2025
@codecov
Copy link

codecov bot commented Oct 31, 2025

Codecov Report

❌ Patch coverage is 86.66667% with 2 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
src/strands/telemetry/tracer.py 86.66% 2 Missing ⚠️

📢 Thoughts on this report? Let us know!

Copy link
Contributor

@poshinchen poshinchen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution.
After the discussion within the team, I believe we should:

  1. Move the attributes to invoke_agent span > gen_ai.tool.definitions
    (semantic conventions link).

  2. Move the flag (include_tool_definitions) into StrandsTelemetry class instead of in the Agent class as this is specifically for the trace.

@Ratish1
Copy link
Contributor Author

Ratish1 commented Nov 4, 2025

Thanks for the contribution. After the discussion within the team, I believe we should:

  1. Move the attributes to invoke_agent span > gen_ai.tool.definitions
    (semantic conventions link).
  2. Move the flag (include_tool_definitions) into StrandsTelemetry class instead of in the Agent class as this is specifically for the trace.

Thanks for the feedback, will change it right away @poshinchen .

@Ratish1
Copy link
Contributor Author

Ratish1 commented Nov 4, 2025

Oh no, I messed up something by force pushing since I was having some rebase problems locally and now it closed the PR automatically. Sorry, could you reopen it or should I create a new PR @poshinchen . Nvm i was able to reopen it.

Copy link
Contributor

@poshinchen poshinchen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, sorry

After a careful read from the otel documentation. We are expecting customers to set multiple values in OTEL_SEMCONV_STABILITY_OPT_IN env var

  • As an example: OTEL_SEMCONV_STABILITY_OPT_IN="gen_ai_latest_experimental,gen_ai_tool_definitions" allows latest semantic conventions, and opt-in gen_ai.tool.definitions.

That being said, could you do the following:

  1. In tracer.py, modify the parsing logic for use_latest_genai_conventions and include_tool_definitions (new).
  2. pass all_tools_config to _start_agent_trace_span and move tool_details = [...] into that method.
  3. I wonder if it's possible to pass them as array?

It’s expected to be an array of objects where each object represents a tool definition. In case a serialized string is available to the instrumentation, the instrumentation SHOULD do the best effort to deserialize it to an array. When recorded on spans, it MAY be recorded as a JSON string if structured format is not supported and SHOULD be recorded in structured form otherwise.

@Ratish1
Copy link
Contributor Author

Ratish1 commented Nov 4, 2025

Hi, sorry

After a careful read from the otel documentation. We are expecting customers to set multiple values in OTEL_SEMCONV_STABILITY_OPT_IN env var

  • As an example: OTEL_SEMCONV_STABILITY_OPT_IN="gen_ai_latest_experimental,gen_ai_tool_definitions" allows latest semantic conventions, and opt-in gen_ai.tool.definitions.

That being said, could you do the following:

  1. In tracer.py, modify the parsing logic for use_latest_genai_conventions and include_tool_definitions (new).
  2. pass all_tools_config to _start_agent_trace_span and move tool_details = [...] into that method.
  3. I wonder if it's possible to pass them as array?

It’s expected to be an array of objects where each object represents a tool definition. In case a serialized string is available to the instrumentation, the instrumentation SHOULD do the best effort to deserialize it to an array. When recorded on spans, it MAY be recorded as a JSON string if structured format is not supported and SHOULD be recorded in structured form otherwise.

No problem , will change it according your feedback. Thanks. Also, regarding your question about passing the data as an array: I looked into it, and you're right that the spec prefers a structured format. However, the OpenTelemetry Python SDK's set_attribute function doesn't have guaranteed support for complex nested objects like a list of dictionaries. What are your thoughts on this, open to being wrong about this.

Copy link
Contributor

@poshinchen poshinchen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you update the description and the commit too?

@Ratish1 Ratish1 changed the title feat(agent): Add opt-in flag to include tool specs in traces for evaluation feat(telemetry): Add tool definitions to traces via semconv opt-in Nov 5, 2025
@Ratish1 Ratish1 requested a review from poshinchen November 5, 2025 15:20
@github-actions github-actions bot added size/m and removed size/m labels Nov 5, 2025
@Ratish1
Copy link
Contributor Author

Ratish1 commented Nov 5, 2025

Could you update the description and the commit too?

I have updated the commit and the description.

Copy link
Contributor

@poshinchen poshinchen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change the tests please

@github-actions github-actions bot added size/m and removed size/m labels Nov 5, 2025
@Ratish1 Ratish1 deployed to manual-approval November 5, 2025 20:17 — with GitHub Actions Active
@Ratish1
Copy link
Contributor Author

Ratish1 commented Nov 5, 2025

Change the tests please

Its changed now, thanks for the help.

@Ratish1 Ratish1 requested a review from poshinchen November 5, 2025 20:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE] Add all tool details under gen_ai.agent.tools attribute

2 participants