feat: add cache_messages to bedrock model

[DayuanJiang]

Description

Introduced the cache_messages configuration parameter to BedrockModel to enable message caching for intermediate steps.

When cache_messages = "default", the agent will add a cache point at the end of the message list each time it calls a Bedrock model. This allows the agent to cache intermediate steps effectively while preserving any existing cache points users may have set.

Key implementation details:

• Adds a new cache point at the end of the last message
• Creates copies of message structures to avoid modifying the original messages passed to format_request

Related Issues

#404 #1015

Documentation PR

None

Type of Change

New feature

Testing

• test_format_request_cache_messages(): Validates that existing cache points are preserved and a new cache point is added at the end

• test_format_request_cache_messages_does_not_modify_original(): Verifies that the original messages passed to format_request are not mutated

  • Tests preservation of existing cache points from multiple messages
  • Verifies correct addition of new cache point at the end of the last message
  • Confirms complete request structure matches expectations
  • Ensures no mutation of original message objects

• I ran hatch run prepare

Checklist

• I have read the CONTRIBUTING document
• I have added any necessary tests that prove my fix is effective or my feature works
• I have updated the documentation accordingly
• I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
• My changes generate no new warnings
• Any dependent changes have been merged and published

---

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[pgrayy]

Can you explain the purpose of clearing cachePoints from previously processed messages?

[DayuanJiang]

Thanks for pointing that out. Originally, the implementation would add a cache point to the last message on each call. To avoid accumulating too many cache points across multiple calls, I decided to remove all existing cache points before adding the new one.

However, I realized that this approach was problematic because it modified the original messages passed to the function. So I modified the code to create a copy of the messages and add the cache point to the copied version, which means clearing existing cachePoints is no longer necessary.

I have also updated the PR description to reflect this change.

[dbschmigelski]

Hi, we are working on #937 where we are introducing support for the more generic SystemContentBlock. I believe this will be a more general solution and we should wait before introducing this bedrock specific logic

[DayuanJiang]

Hi @dbschmigelski, thanks for responding!

I appreciate the heads up about #937, but I believe SystemContentBlock is actually addressing a different concern from this PR.

SystemContentBlock is specifically for handling system prompts.
This PR, on the other hand, is focused on message caching - specifically adding cache points to the message list (the conversation history between user and assistant) during agent operations.

While both involve caching, they operate on different parts of the request:

• SystemContentBlock → system prompt caching that have being done by using "cache_prompt"
• This PR → message list caching by using "cache_messages"

These are complementary features rather than overlapping ones, so I don't think we need to wait for #937 to merge this. That said, happy to discuss if you see a specific conflict I am missing!

[dbschmigelski]

Hi @dbschmigelski, thanks for responding!

I appreciate the heads up about #937, but I believe SystemContentBlock is actually addressing a different concern from this PR.

SystemContentBlock is specifically for handling system prompts. This PR, on the other hand, is focused on message caching - specifically adding cache points to the message list (the conversation history between user and assistant) during agent operations.

While both involve caching, they operate on different parts of the request:

• SystemContentBlock → system prompt caching that have being done by using "cache_prompt"
• This PR → message list caching by using "cache_messages"

These are complementary features rather than overlapping ones, so I don't think we need to wait for #937 to merge this. That said, happy to discuss if you see a specific conflict I am missing!

Hi, this is technically true. But, the reason I want to wait is because adding the SystemContentBlock marks a transition from a Bedrock specific syntax of

bedrock_model = BedrockModel(
    model_id="anthropic.claude-sonnet-4-20250514-v1:0",
    cache_prompt="default"
)

to a model agnostic one. I believe we can do something similar for cache_messages which is why I want to hold the discussion until we familiarize ourselves general approach. What I want to avoid is introducing cache_messages and then we get a follow up asking for this in another model when we could have solved message caching in the general case from the start

[DayuanJiang]

Good point, is there a timeline for the general approach?

[dbschmigelski]

Good point, is there a timeline for the general approach?

Yup, the PR for #937 will be raised this week

[dbschmigelski]

Hi, following up with that #937 is being addressed in #1112. There will be a follow up PR specifically for the LiteLLM portion.