AI 智能体的「土拨鼠之日」：在 Amazon Bedrock 上使用 OpenAI 模型的提示词缓存模式 | Amazon Web Services

每次启动你的智能体，它在执行操作前都必须重新阅读自己的全部经历。下面介绍如何让它记住这些内容，而不必再向它重复提供相同的上下文。Every time you spin off your agent, it has to re-read its whole life story before performing an action. Here’s how to make it remember without having to feed it the same context again.

在电影《土拨鼠之日》中，比尔·默里的角色每天都会重新经历同一个2月2日。他一次又一次地在同一家旅馆、同一座小镇醒来，收音机里播放着同一首歌；一些影迷估计，这样的日子持续了34年。In the movie Groundhog Day, Bill Murray’s character re-lives the same February 2 every day. He wakes up to the same song playing on the radio, in the same inn, and the same small town, again and again for what some movie fans estimate to be 34 years.

除非他从头重新推演，否则前一天完成的一切都不会延续下来。AI 智能体仿佛活在《土拨鼠之日》的循环里。每当智能体迈出一步（调用工具、读取结果、决定接下来做什么），为了保持连贯性，它都必须把截至目前的完整经历发送给模型：工具定义、指令、此前的每一次工具调用，以及每一个结果。模型只会根据你提供的输入来预测下一个 token。它在请求之间没有内部记忆，因此每次都必须把所有内容重新读一遍。一次编程会话进行到第 20 轮时，模型会重新阅读第 1 到第 19 轮，而这些 token 每一次都要由你反复付费。Nothing he accomplished the day before carries over unless he works it out again from scratch. AI agents live in a version of Groundhog Day. Every time an agent takes a step (calls a tool, reads the result, decides what to do next), it has to send the model its entire story so far to maintain continuity: the tool definitions, the instructions, every previous tool call, and every result. The model only predicts the next token based on the input you provide. It has no internal memory between requests, so it must read all of it again. By turn 20 of a coding session, the model re-reads turns 1–19, and it’s you who pays for every one of those tokens, again and again.

提示缓存打破了这一循环。服务会存储你的输入提示（指令、工具和内容），并在下一次请求时重复使用，输入成本仅为原价的一小部分。Prompt caching breaks the cycle. The service stores your input prompt (instructions, tools, and content) and reuses it on the next request, at a fraction of the input price.

在这篇文章中，我将带你了解一个智能体从首次运行，到获得新工具并生成辅助智能体的整个过程。期间，我还会向你展示提示缓存如何降低成本，并让智能体保持响应迅速和行为一致。In this post, I walk you through an agent’s life, from its first turn to when it gets new tools and spawns helpers. Along the way, I show you how prompt caching keeps costs down and keeps agents responsive and consistent.

智能体发送给其 LLM 的内容What an agent sends to its LLM

在讨论缓存之前，先看看典型智能体首次调用大型语言模型（LLM）时包含哪些内容：Before talking about caching, it helps to look at what a typical agent’s first invocation to the large language model (LLM) contains:

• 工具列表——代理可调用的所有函数的定义。发布新版本时会发生变化。
  Tool list – The definitions of every function the agent can call. Changes when you ship a new release.
• 说明——开发者提示：规则、角色设定、政策和示例。也会随版本发布而变化。
  Instructions – The developer prompt: rules, persona, policies, examples. Also changes per release.
• 历史记录——截至目前本次会话中发生的一切：任务、模型的工具调用以及工具结果（内容会随着每一轮对话不断增加）。
  History – Everything that happened in this session so far: the task, the model’s tool calls, and the tool results (which grows every turn).
• 新一轮 – 最新的工具结果或用户消息（每次可能都不同）。
  New turn – The latest tool result or user message (which might be different every time).

如果不设置缓存，代理每一轮都会发送给模型的内容：What an agent sends to the model, turn after turn if you don’t set up caching:

图 1：智能体每轮向模型发送的内容Figure 1: What an agent sends to the model on every turn

提示缓存在底层是如何工作的How prompt caching works under the hood

模型读取提示时，首先会执行一个称为预填充（prefill）的写入过程。在预填充期间，它会处理每个输入 token，并构建键向量和值向量（也称为键值向量，或 KV），这些向量构成模型的工作状态。这一过程发生在模型生成第一个 token 之前。When a model reads a prompt, it first does a write called prefill. During prefill, it processes every input token and builds its key and value vectors (also known as key-value, or KV), which form the model’s working state. This happens before the model writes its first token.

提示缓存会保存该状态，其中包含系统提示、工具、内容、历史记录等。当后续请求以相同的输入令牌（即相同文本）开头时，提示缓存服务会加载已保存的状态，而不是重新计算。随后，它只处理下一个负载中新增的内容，再将其传递给模型。提示缓存可以是隐式的，即由 Amazon Bedrock 选择可复用前缀的结束位置；也可以是显式的，即由你的代码使用 prompt_cache_breakpoint 标记该位置。你可以同时使用这两种方式，我们也建议这样做。Prompt caching saves that state, which contains the system prompt, tools, content, history, and so on. When a later request starts with the same input tokens (that is, the same text), the prompt caching service loads the saved state instead of computing it again. It then processes only what is new in the next payload to the model. Prompt caching can be implicit, where Amazon Bedrock selects where the reusable prefix ends, or explicit, where your code marks that endpoint using prompt_cache_breakpoint. You can use both approaches together, and we recommend doing so.

第1轮与第2轮相比Turn 1 compared to turn 2

图 2：第 1 轮将提示前缀保存到断点处；第 2 轮复用该前缀，仅需为新问题支付全额费用。Figure 2: Turn 1 saves the prompt prefix up to the breakpoint, and turn 2 reuses it and pays full price only for the new question

隐式还是显式？为什么两者都需要？Implicit or explicit, and why do we need both?

对于智能体工作负载，隐式缓存可让 Bedrock 随着对话不断延长，自行决定缓存哪些内容。借助显式缓存，你可以将工具和指令等共享内容标记为可在不同会话间复用，也可在启动辅助智能体时复用。你不一定总要同时使用这两种缓存方式，但将它们结合起来，既能提供可复用的起点，也能让 Bedrock 自动管理不断增长的对话。For agentic workloads, implicit caching lets Bedrock choose what to cache as the conversation grows. With explicit caching, you can mark shared content, such as tools and instructions, for reuse across sessions and when spinning up helper agents. You don’t always need both forms of caching, but combining them gives you a reusable starting point while Bedrock manages the growing conversation automatically.

新会话也可以复用这个共享起点，同时每段对话都基于各自缓存的历史继续展开。本文将从第一轮开始，随后展示如何在后续轮次、新工具以及子代理之间传递缓存内容。New sessions can also reuse that shared starting point while each conversation builds on its own cached history. In this post, we start with the first turn, then show how to carry cached content through later turns, new tools, and across subagents.

你好，缓存世界Hello cached world

首先要做的是缓存提示词中的静态部分：即在各个请求之间保持不变的工具和指令。这些示例使用你现有的 Amazon Bedrock 客户端、TOOLS 和任务。请将缩短版的 INSTRUCTIONS 字符串替换为实际的策略和示例。可复用前缀必须达到 1,024 个 token 才符合条件。The first win is to cache the static part of the prompt: the tools and instructions that stay the same across requests. These examples use your existing Amazon Bedrock client, TOOLS, and task. Replace the shortened INSTRUCTIONS string with your real policies and examples. The reusable prefix must reach 1,024 tokens to qualify.

缓存静态内容时，请使用显式模式：选择已保存前缀的结束位置。When caching static content, use explicit mode: choose where the saved prefix ends.

KEY = "support-agent:v7:gpt-6-astra"
INSTRUCTIONS = "Your full support policies and examples..."

# Rendered context: [provider context][TOOLS][INSTRUCTIONS] | [task]
# Cache starts at the beginning; the marker below chooses its end.
resp = client.responses.create(
    model="us.openai.gpt-6-astra",
    store=False,  # Response storage; prompt caching still works.
    include=["reasoning.encrypted_content"],
    prompt_cache_key=KEY,
    prompt_cache_options={"mode": "explicit", "ttl": "30m"},
    tools=TOOLS,  # Rendered before instructions; keep order stable.
    input=[
        {
            "role": "developer",
            "content": [{
                "type": "input_text",
                "text": INSTRUCTIONS,
                # Save through this whole block, including TOOLS.
                "prompt_cache_breakpoint": {"mode": "explicit"},
            }],
        },
        # User message is processed but not cached.
        {"role": "user", "content": task},
    ],
)

prompt_cache_key 用于告知服务哪些请求应共享同一个缓存。稍后讲到在新会话中读取缓存时，我们会再讨论如何使用它。在我们的一项显式缓存测试中，约 4,000 个 token 的工具与指令前缀在首次请求时写入了 4,120 个 token，后续请求则读取了这 4,120 个 token。任务（或用户请求）位于缓存边界之后，因此不会触发缓存写入。The prompt_cache_key tells the service which requests are meant to share a cache. We come back to how to use it when we get to reading the cache in new sessions. In one of our tests using explicit caching, the roughly 4,000-token tools-and-instructions prefix wrote 4,120 tokens on the first request and read those 4,120 tokens on later requests. The task (or user request) comes after the cache boundary, so this doesn’t trigger a write to the cache.

请记住，提示缓存只有在传入完全相同的文本时才会生效。务必确保每次都以相同的方式序列化智能体的指令和工具列表。Model Context Protocol (MCP) 服务器及其他形式的工具传递方式都需要进行索引，以确保它们保持一致。将任何会变化的内容（例如日期、用户名或请求 ID）放在缓存指令之后。Keep in mind that prompt caching works only when you pass exactly the same text. Make sure that the agent’s instructions and tool lists are serialized the same way every time. Model Context Protocol (MCP) servers and other forms of tool delivery need to be indexed to make sure they stay consistent. Keep anything that varies, such as the date, the user’s name, or a request ID, after the cached instructions.

随着内容不断增长并变得多样化（涉及不同的工具和内容），请使用隐式模式，让服务根据对话自动处理。这里，第一个请求没有显式标记。它最新的符合条件的消息是用户的任务，因此自动缓存断点会位于该任务之后，并将工具、指令和任务一并纳入缓存。为便于说明，下面的示例显式展示了该模式，尽管隐式模式是默认设置。将 store=False 会使对话历史保留在你的应用程序中，但不会禁用缓存。As the content grows and varies (different tools and content), use implicit mode to let the service follow the conversation. Here, the first request has no explicit marker. Its latest eligible message is the user’s task, so the automatic breakpoint comes after that task and covers the tools, instructions, and task together. The mode is shown explicitly in the following example for clarity, even though implicit is the default. Setting store=False leaves conversation history in your application. It doesn’t disable caching.

def ask_with_implicit_cache(history):
    return client.responses.create(
        model="us.openai.gpt-6-astra",
        store=False,
        include=["reasoning.encrypted_content"],
        prompt_cache_key=KEY,
        prompt_cache_options={"mode": "implicit", "ttl": "30m"},
        tools=TOOLS,
        input=history,  # Everything gets cached
    )

history = [
    {"role": "developer", "content": INSTRUCTIONS},
    {"role": "user", "content": task},
]
# Automatic endpoint: the end of the user message.
resp = ask_with_implicit_cache(history)

当下一轮继续执行同一任务时，这很有用。不同的任务与旧任务的前缀不匹配，而隐式写入不一定会仅为工具和指令单独创建条目。需要让这个较短的共享前缀可跨任务复用时，请在那里使用显式标记。This is useful when the next turn continues the same task. A different task doesn’t match the prefix through the old task, and the implicit write doesn’t necessarily create a separate entry for only the tools and instructions. Use an explicit marker there when you need that shorter shared prefix to be reusable across tasks.

当代理需要一个共享起点和不断增长的历史记录时，两者都要使用。从以下代码示例中带标记的 developer 块开始，并调用函数 ask_with_implicit_cache。请求级模式仍保持隐式。块级标记会在指令之后添加一个显式保存点。服务也可以通过最新的符合条件的消息进行保存。这样，在对话不断扩展的同时，共享前缀仍可供使用。Use both when the agent needs a shared starting point and a growing history. Start with the marked developer block in the following code example and call the function ask_with_implicit_cache. The request-level mode stays implicit. The block-level marker adds an explicit save point after the instructions. The service can also save through the latest eligible message. This keeps the shared prefix available while the conversation grows.

history = [
    {
        "role": "developer",
        "content": [{
            "type": "input_text",
            "text": INSTRUCTIONS,
            "prompt_cache_breakpoint": {"mode": "explicit"}, 
        }],
    },
    {"role": "user", "content": task},
]
# Still implicit mode, now with an explicit startup save point in the history.
resp = ask_with_implicit_cache(history)

缓存对话历史记录和工具结果Caching conversation history and tool results

模型会按名称请求工具并提供相应参数。你的应用必须执行这些工具。execute_function 是你构建的辅助函数，用于将这些请求路由到匹配的 Python 函数并返回其结果。下面的循环使用此辅助函数运行每个请求的工具。它会将模型的输出和所有工具结果追加到 history 列表，然后再次发送该列表，直到模型返回最终答案。在隐式模式下，Amazon Bedrock 可以复用未更改的前缀，并将新增内容缓存到最后一个工具结果。保持较早的消息和初始 TOOLS 不变。The model requests tools by name and supplies their arguments. Your application must execute them. execute_function is a helper you build to route those requests to the matching Python functions and return their results. The following loop uses this helper to run each requested tool. It appends the model’s output and all tool results to the history list, then sends that list again until the model returns a final answer. In implicit mode, Amazon Bedrock can reuse the unchanged prefix and cache the added content through the last tool result. Keep earlier messages and the initial TOOLS unchanged.

import json

# Map tool names to their Python implementations.
TOOL_HANDLERS = {
    "get_owner": get_owner,
    # Register your other initial tools here.
}

def execute_function(name, arguments):
    """Run the requested tool and return its result."""
    handler = TOOL_HANDLERS.get(name)
    if handler is None:
        return {"error": f"Unknown tool: {name}"}
    return handler(**arguments)

while True:
    # Preserve all output items, including encrypted reasoning.
    history.extend(resp.output)
    calls = [x for x in resp.output if x.type == "function_call"]
    if not calls:
        break
    for call in calls:
        result = execute_function(call.name, json.loads(call.arguments))
        history.append({
            "type": "function_call_output",
            "call_id": call.call_id,
            "output": json.dumps(result),
        })
    # The last result in this group supplies the automatic endpoint.
    resp = ask_with_implicit_cache(history)

对于后续操作，请追加一条新的用户消息，然后再次调用该辅助函数。保持之前的消息不变，以便继续使用其缓存端点。For follow-ups, append a new user message and call the helper again. Keep earlier messages unchanged so their cached endpoints remain available.

滚动式断点：每一轮都会读取此前的所有内容Rolling breakpoints: Each turn reads everything before it

图 3：滚动式断点，每一轮都会读取此前的所有内容，并且只写入自己的工具结果Figure 3: Rolling breakpoints, where each turn reads everything before it and writes only its own tool result

对于单独的仅显式运行，请从组合版 Hello cached world 示例中重建这段包含两条消息的历史记录。此循环会将工具结果包装在 input_text 块中，并标记每个批次中的最终结果。较早的标记会保留在历史记录中。请参阅当前针对长对话的查找规则。For a separate explicit-only run, recreate the two-message history from the combined Hello cached world example. This loop wraps tool results in input_text blocks and marks the final result in each batch. Earlier markers stay in history. See the current lookup rules for long conversations.

import json

while True:
    resp = client.responses.create(
        model="us.openai.gpt-6-astra", store=False,
        include=["reasoning.encrypted_content"],
        tools=TOOLS, prompt_cache_key=KEY, input=history,
        prompt_cache_options={"mode": "explicit", "ttl": "30m"},
    )
    d = resp.usage.input_tokens_details
    print("read", d.cached_tokens, "write", d.cache_write_tokens)
    history.extend(resp.output)  # Includes encrypted reasoning.
    calls = [x for x in resp.output if x.type == "function_call"]
    if not calls:
        break
    batch = []
    for call in calls:
        result = execute_function(call.name, json.loads(call.arguments))
        batch.append({
            "type": "function_call_output", "call_id": call.call_id,
            "output": [{"type": "input_text", "text": json.dumps(result)}],
        })
    # Add the newest boundary; retain every earlier marker.
    batch[-1]["output"][-1]["prompt_cache_breakpoint"] = {"mode": "explicit"}
    history.extend(batch)

在这项包含六轮对话的工作负载中，GPT-6 Astra 的隐式缓存效果并列最佳：与不使用缓存相比，输入成本降低了约 62%。在同一模型上，显式滚动断点节省了约 61%。随着对话不断推进，两种方式都复用了更多历史内容。这些是该工作负载的测试结果。请测量自己的智能体实际节省了多少。GPT-6 Astra tied for the best implicit result in this six-turn workload: about 62 percent lower input cost than running without caching. Explicit rolling breakpoints saved about 61 percent on the same model. Both reused more history as the conversation grew. These are workload results. Measure your own agent’s savings.

图 4：GPT-6 Astra 上各种缓存设置的输入成本，相对于未启用缓存的情况Figure 4: Input cost of every caching setup on GPT-6 Astra, relative to no caching

注意：Watch out for:

• 仅在仅显式模式下设置静态断点——工具和指令会被缓存，但不断增长的历史记录会在每一轮重新处理（0.61）。这一限制不适用于组合示例；该示例还会在符合条件的对话结束时保存缓存。
  Only a static breakpoint in explicit-only mode – The tools and instructions are cached, but the growing history is processed again on every turn (0.61). This limitation doesn’t apply to the combined example, which also saves through eligible conversation endings.
• 移动单个断点可能会阻止缓存复用，并导致仅显式模式下重复写入。也保留上一轮的断点。
  Moving a single breakpoint can prevent cache reuse and cause repeated writes in explicit-only mode. Keep the previous turn’s breakpoint too.
• 仅通过变化的任务进行隐式写入——新任务可能会共享工具和指令，但不会共享之前保存的端点。如果不同任务需要复用较短的前缀，请在共享指令后添加显式标记。
  Only an implicit write through a changing task – A new task might share the tools and instructions but not the previously saved endpoint. Add an explicit marker after the shared instructions if different tasks need to reuse that shorter prefix.

当工具发生变化时When tools change

有时，智能体的工具列表会发生变化，或新增工具。请保持初始列表稳定：添加、移除或重新排列定义都可能导致缓存前缀失效。根据工具的可用方式选择合适的方法。Sometimes an agent’s tool list changes, or new tools get added. Keep the initial list stable: adding, removing, or reordering definitions can invalidate the cached prefix. Choose the approach that fits how the tool becomes available.

当已知工具在代理流程的不同步骤中被调用时：发送相同的已排序 TOOLS，并使用 allowed_tools 限制模型可调用的工具。对于 allowed_tools，"mode": "auto" 允许模型自行决定是否调用某个允许的工具，而 "mode": "required" 则要求至少调用一个允许的工具。将其作为 tool_choice 传递。此限制不会更改缓存的目录：When known tools are invoked in different steps of the agent’s flow: Send the same sorted TOOLS and use allowed_tools to limit what the model can call. With allowed_tools, "mode": "auto" lets the model decide whether to call an allowed tool, and "mode": "required" requires at least one allowed tool call. Pass this as tool_choice. The restriction doesn’t change the cached catalog:

tool_choice={"type": "allowed_tools", "mode": "auto",
             "tools": [{"type": "function", "name": "get_owner"}]}

会话期间新增工具。Amazon Bedrock Runtime 和 Amazon Bedrock Mantle 端点均接受一种名为 additional_tools 的输入项类型，适用于 GPT-6 Astra、GPT-6.1 Sol 和 GPT-5.6 Luna。有关各 AWS 区域的模型可用情况，请参阅 Amazon Bedrock 中的“按 AWS 区域划分的支持模型”。在辅助函数 execute_function 中注册新工具 get_due_date，然后将其架构追加到智能体循环生成的完整历史记录中。保持初始 TOOLS 不变：New tools during a session. Both Amazon Bedrock Runtime and Amazon Bedrock Mantle endpoints accept an input item type called additional_tools for GPT-6 Astra, GPT-6.1 Sol, and GPT-5.6 Luna. For model availability by AWS Region, see Supported models by AWS Region in Amazon Bedrock. Register the new tool get_due_date in the helper function execute_function, then append its schema to the completed history from the agent loop. Keep the initial TOOLS unchanged:

TOOL_HANDLERS["get_due_date"] = get_due_date
NEW_TOOL = {
    "type": "function", "name": "get_due_date",
    "description": "Return the due date for a support case.",
    "strict": True,
    "parameters": {
        "type": "object",
        "properties": {"case_id": {"type": "string"}},
        "required": ["case_id"], "additionalProperties": False,
    },
}
history.append({
    "type": "additional_tools", "role": "developer",
    "tools": [NEW_TOOL],
})
history.append({
    "role": "user", "content": "Find the due date for case C-123.",
})
# Implicit mode saves through this new user message, including NEW_TOOL.
resp = ask_with_implicit_cache(history)
d = resp.usage.input_tokens_details
print("old prefix read", d.cached_tokens, "new write", d.cache_write_tokens)

# Example output:
# old prefix read 2048 new write 256

一个新工具在会话中途加入了A new tool arrives mid-session

图 5：追加工具可使先前的前缀继续复用，示例数量来自 Amazon Bedrock Runtime 上的 GPT-6 AstraFigure 5: Appending a tool keeps the earlier prefix reusable, with example counts from GPT-6 Astra on Amazon Bedrock Runtime

使用新的响应继续工具执行循环。该循环会将模型的输出添加到历史记录中，运行所有请求的工具，并追加工具返回的结果。后续请求中要保留 additional_tools 项，以便新工具仍可用。在隐式模式下，Amazon Bedrock 可以缓存到新的用户消息，包括该消息之前的工具定义。在仅显式模式下，保留之前的缓存断点，并在该用户消息中的一个 input_text 块上添加一个断点。additional_tools 项本身不能设置断点。Continue the tool-execution loop with the new response. The loop adds the model’s output to history, runs any requested tools, and appends their results. Keep the additional_tools item in later requests so the new tool remains available. In implicit mode, Amazon Bedrock can cache through the new user message, including the tool definitions that precede it. In explicit-only mode, retain earlier breakpoints and add one to an input_text block in that user message. The additional_tools item itself can’t hold a breakpoint.

不同的角色，不同的工具目录。当不同的智能体角色需要使用不同工具时，请在每次会话开始时选择一个稳定的工具配置，例如支持或计费。每个配置都有自己的缓存前缀。标记共享指令，以便之后使用该配置的任务可以复用它们，然后随着对话的推进，使用任一模式。Different roles, different catalogs. When different agent roles need different tools, choose a stable tool profile, such as support or billing, at the start of each session. Each profile gets its own cached prefix. Mark the shared instructions so later tasks with that profile can reuse them, then use either mode as the conversation grows.

缓存的工具配置文件Cached tool profiles

图 6：每个工具配置文件都有自己的缓存前缀，可供匹配的会话重复使用。Figure 6: Each tool profile has its own cached prefix, reusable by matching sessions

子代理与旁路调用Subagents and side calls

委派任务不应让每个助手都重复读取相同的背景信息。共享前缀，然后在末尾添加每个助手的任务。请注意，在 AI 智能体中使用缓存时，工具和系统提示可能会泄露或遭到利用。建立负责任的 AI 实践，并将提示缓存的实现纳入相关治理。Delegating shouldn’t make every helper reread the same background. Share the prefix, then add each helper’s task at the end. Be aware that when you use caching with AI agents, tools and system prompts can be leaked or exploited. Establish responsible AI practices that include your prompt caching implementation as part of that governance.

先预热一次，然后分发给多个任务。先发送一个普通请求，并等待它完成后再启动辅助任务。即使模式是隐含的，也要在每个请求中明确标注共享上下文：Warm once, then fan out. Send an ordinary request first and wait for it to finish before launching helpers. Mark the shared context explicitly in every request, even when the mode is implicit:

shared = {"role": "developer", "content": [{
    "type": "input_text", "text": INSTRUCTIONS,
    "prompt_cache_breakpoint": {"mode": "explicit"},
}]}
warm = client.responses.create(
    model="us.openai.gpt-6-astra", store=False,
    tools=TOOLS, prompt_cache_key=KEY,
    prompt_cache_options={"mode": "implicit", "ttl": "30m"},
    max_output_tokens=16,
    input=[shared, {"role": "user", "content": "Reply READY."}],
)
d = warm.usage.input_tokens_details
if not (d.cache_write_tokens or d.cached_tokens):
    raise RuntimeError("Shared prefix was not cached.")
from concurrent.futures import ThreadPoolExecutor

# helper_tasks contains the task text for each helper.
helper_inputs = [
    [shared, {"role": "user", "content": task}]
    for task in helper_tasks
]
with ThreadPoolExecutor(max_workers=4) as pool:
    helpers = list(pool.map(ask_with_implicit_cache, helper_inputs))

这些辅助程序使用与预热相同的 TOOLS、模型和 KEY。即使预热在达到输出上限时结束，仍然可以填充缓存。用量检查会在辅助程序启动前确认缓存已生效。The helpers use the same TOOLS, model, and KEY as the warm-up. A warm-up that ends at the output cap can still populate the cache. The usage check confirms cache activity before the helpers start.

子代理：先预热一次共享前缀，然后并行分发任务Subagents: Warm the shared prefix once, then fan out

图 7：启动辅助进程前先预热共享前缀，然后每个辅助进程再添加自己的任务。Figure 7: Warm the shared prefix before launching helpers, and each helper then adds its own task

让旁路调用使用相同的开头。智能体循环会将每个 `resp.output` 及其对应的工具结果（包括加密的推理内容）追加到历史记录中。父调用结束后，将这份完整列表复制给摘要器或评判器。相同的辅助函数会保留原有的工具、模型和密钥：Give side calls the same beginning. The agent loop builds history by appending every resp.output and its matching tool results, including encrypted reasoning. After the parent finishes, copy that complete list for a summarizer or judge. The same helper keeps the tools, model, and key unchanged:

from copy import deepcopy

# history is the completed history built by the agent loop above.
# Copy it so the helper can add instructions without changing the parent.
helper_history = deepcopy(history)
helper_history.append({
    "role": "developer",
    "content": "Summarize this conversation in one sentence. Do not call tools.",
})
summary = ask_with_implicit_cache(helper_history)
d = summary.usage.input_tokens_details
print("history read", d.cached_tokens, "write", d.cache_write_tokens)
print(summary.output_text)

# Example output (assumed values and conversation):
# history read 4096 write 32
# Case C-123 is due on October 31, 2026.

helper 可以复用历史记录中已缓存的部分，但追加的 developer 消息不会自动扩展缓存以包含摘要指令。如果后续请求会复用相同的历史记录和指令，请将指令放在带有 prompt_cache_breakpoint 的 input_text 块中。对于一次性摘要，无需额外标记。The helper can reuse the cached portion of history, but the appended developer message doesn’t automatically extend the cache to include the summary instruction. If later requests reuse the same history and instruction, place the instruction in an input_text block with prompt_cache_breakpoint. For a one-time summary, no additional marker is needed.

暂停、重新启动与缓存键Pauses, restarts, and the cache key

暂停。缓存前缀的存活时间（TTL）至少为 30 分钟，成功复用后会刷新，并且可能保留更长时间。如果会话很快恢复，常规预热可以在前缀过期前刷新它。请比较该请求的成本与任由缓存过期的成本。有关 TTL 规则，请参阅 Amazon Bedrock 文档。Pauses. Cached prefixes have a minimum 30-minute time to live (TTL), refreshed by successful reuse, and might remain longer. If a session resumes soon, an ordinary warm-up can refresh the prefix before it expires. Compare that request’s cost with letting the cache expire. For the TTL rules, see the Amazon Bedrock documentation.

新会话。对于旨在共享前缀的请求，请保持 prompt_cache_key 稳定。即使各会话的提示词相同，会话专属的键也会将它们隔离开来。在下面的示例中，切换到共享键后，各会话便可以复用该前缀。New sessions. Keep prompt_cache_key stable for requests that are meant to share a prefix. A session-specific key isolates sessions even when their prompts match. In the following example, switching to a shared key let sessions reuse the prefix.

只有共享同一个键的新会话才会复用缓存。New sessions reuse the cache only if they share the key

图 8：会话专属密钥可区分缓存条目，而共享密钥可让匹配的会话重复使用这些条目Figure 8: A session-specific key separates cache entries, and a shared key lets matching sessions reuse them

复用现有缓存内容需要使用相同的模型、AWS 账户和区域，并且存在可用的匹配前缀。以下示例将显式启动标记与隐式模式结合使用。每个会话都以相同的指令和各自的任务开始。两者使用相同的计算键：Reuse of existing cache content requires the same model, AWS account, and Region, and an available matching prefix. The following example combines an explicit startup marker with implicit mode. Each session starts with the same instructions and its own task. Both use the same computed key:

import hashlib

def cache_key(app, model, prompt_version, instructions):
    digest = hashlib.sha256(instructions.encode()).hexdigest()[:12]
    return f"{app}:{prompt_version}:{model}:{digest}"

KEY = cache_key("support-agent", "us.openai.gpt-6-astra", "v7", INSTRUCTIONS)
prefix = {"role": "developer", "content": [{
    "type": "input_text", "text": INSTRUCTIONS,
    "prompt_cache_breakpoint": {"mode": "explicit"},
}]}
for label, question in [
    ("session A", "Explain the refund policy without calling tools."),
    ("session B", "Explain the escalation policy without calling tools."),
]:
    # A fresh conversation for each session; the startup prefix matches.
    session_history = [prefix, {"role": "user", "content": question}]
    result = ask_with_implicit_cache(session_history)
    d = result.usage.input_tokens_details
    print(label, "read", d.cached_tokens, "write", d.cache_write_tokens)

在冷启动时，会话 1 写入共享前缀。即使会话 2 执行的是不同任务，也可以读取该前缀。每个任务也可以隐式缓存。查看使用计数器：匹配的键和标记使复用成为可能，但仍需有可用的缓存条目。On a cold start, session 1 writes the shared prefix. Session 2 can read it even though its task can be different. Each task can also be cached implicitly. Check the usage counters: matching keys and markers make reuse possible, but a cache entry still has to be available.

提示词缓存带来的效果What prompt caching delivers

成本。对于一个由 GPT-6 Astra 驱动、启动前缀为 8,000 个 token、并经历六轮（每轮增加 2,000 个 token）的智能体，一次性静态缓存将成本降至 0.50 美元；滚动断点则将模型估算的输入成本降低了 64%，从每个会话 0.87 美元降至 0.32 美元。Cost. For a GPT-6 Astra powered agent with an 8,000-token startup prefix and six turns adding 2,000 tokens each, one-time static caching reduced cost to $0.50, and rolling breakpoints reduced modeled input cost by 64 percent: from $0.87 to $0.32 per session.

图 9：支持代理示例中每个会话的累计输入成本，按 Amazon Bedrock 上 GPT-6 Astra 的按需价格计算Figure 9: Cumulative input cost per session for the support-agent example, using Amazon Bedrock on-demand prices for GPT-6 Astra

如何判断缓存是否正常工作How to tell caching is working

先从响应中的用量信息入手。cached_tokens 表示从缓存读取的输入量，cache_write_tokens 表示写入缓存的输入量。随着对话不断增长，请检查后续轮次是否复用了先前的对话历史，以及新的写入是否反映了新增内容。Start with response usage. cached_tokens reports input read from the cache, and cache_write_tokens reports input written to it. As the conversation grows, check that later turns reuse earlier history and that new writes reflect the content being added.

对于 Amazon Bedrock Runtime 端点的生产监控，请在 Amazon CloudWatch 中使用 CacheReadInputTokenCount 和 CacheWriteInputTokenCount 指标。需要单条请求记录时，请启用模型调用日志记录，然后使用 Amazon CloudWatch Logs Insights 查询这些记录。对于 Amazon Bedrock Mantle 端点，请在应用程序日志中记录响应中的 usage 字段。其 Amazon CloudWatch 命名空间目前没有单独的缓存计数器。For production monitoring on the Amazon Bedrock Runtime endpoint, use the CacheReadInputTokenCount and CacheWriteInputTokenCount metrics in Amazon CloudWatch. Enable model invocation logging when you need individual request records, then query them with Amazon CloudWatch Logs Insights. For the Amazon Bedrock Mantle endpoint, record the response usage fields in your application logs. Its Amazon CloudWatch namespace currently has no separate cache counters.

要查看成本，请在 AWS Cost Explorer 中筛选 Amazon Bedrock，并按 Usage type 分组。每个模型的缓存读取、缓存写入和未缓存输入都会显示为单独的条目。计算它们在输入成本中的占比时，请排除输出 token 条目。To check costs, filter AWS Cost Explorer to Amazon Bedrock and group by Usage type. Cache reads, cache writes, and uncached input appear as separate entries per model. Exclude output-token entries when calculating their shares of input cost.

一份简短的检查清单A short checklist

• 保持可复用前缀稳定。先放置共享工具和指令，再追加对话历史和新轮次。更改时间戳、消息或工具定义后，超出该更改部分的内容将无法复用。如果较短的前缀有自己的已保存终点，仍然可以复用。
  Keep the reusable prefix stable. Put shared tools and instructions first, then append conversation history and new turns. Changing a timestamp, message, or tool definition prevents reuse beyond that change. A shorter prefix can still be reused if it has its own saved endpoint.
• 围绕你想要重复使用的内容选择边界。对于不断增长的历史记录，可以使用隐式缓存；对于共享前缀，可以使用显式标记；也可以两者兼用。仅显式模式可避免写入可丢弃的后缀。扩展仅显式模式的对话时，请保留之前的标记，以便后续请求找到已保存的前缀。
  Choose boundaries around content you want to reuse. Use implicit caching for growing history, explicit markers for shared prefixes, or both. Explicit-only mode avoids writing disposable suffixes. When extending an explicit-only conversation, retain earlier markers so later requests can find the saved prefixes.
• 保持初始工具目录不变。使用 allowed_tools 限制可调用的工具。通过 additional_tools 添加已知定义，或通过带命名空间的客户端 tool_search 发现工具。如果不同的代理角色需要不同的工具目录，请选择稳定的工具配置。
  Keep the initial tool catalog unchanged. Use allowed_tools to restrict which tools are callable. Append known definitions with additional_tools, or discover them through namespaced client-side tool_search. Choose a stable tool profile when different agent roles need different catalogs.
• 检查前缀大小和有效期。可缓存的前缀必须至少包含 1,024 个令牌。缓存的前缀在最近一次写入或成功复用后的至少 30 分钟内仍可复用。
  Check prefix size and lifetime. A cacheable prefix must contain at least 1,024 tokens. Cached prefixes remain eligible for reuse for at least 30 minutes after their latest write or successful reuse.
• 启动辅助程序前，先预热共享内容。在预热请求和辅助程序请求中都标记共享前缀，包括使用隐式模式时。并行调用开始前，先完成预热请求。
  Warm shared content before launching helpers. Mark the shared prefix in both the warm-up and helper requests, including when using implicit mode. Finish the warm-up request before starting parallel calls.
• 在每个共享组内保持缓存键稳定。旨在复用前缀的请求必须使用相同的键。使用不同的键可以让不同客户或用户的缓存统计保持独立。
  Keep cache keys stable within each sharing group. Requests intended to reuse a prefix must use the same key. Separate keys can keep cache accounting distinct between customers or users.
• 衡量整个工作流中的节省情况。缓存写入费用为正常输入费率的 1.25 倍。读取价格取决于模型。跟踪写入成本和读取节省额：开发期间使用响应中的用量数据，生产环境中使用监控和账单数据。
  Measure savings across the workflow. Cache writes cost 1.25 times the normal input rate. Read prices depend on the model. Track write costs alongside read savings, using response usage during development and monitoring and billing data in production.

2月3日醒来Waking up on February 3rd

比尔·默里只有在不再重复同样的错误，并开始把每天学到的东西带到下一天时，才能逃离《土拨鼠之日》的循环。智能体也能通过类似的方式做到这一点，只不过没那么浪漫：随着历史记录不断增长，保持提示词开头逐字节稳定，并设置自动或显式的缓存边界。Bill Murray only escapes Groundhog Day once he stops repeating the same mistakes and starts carrying what he learned from one day to the next. Agents get there in a similar, if less romantic, way: by keeping the start of their prompt byte-for-byte stable and using automatic or explicit cache boundaries as the history grows.

本文介绍了如何缓存共享工具和指令、在多个代理轮次中扩展缓存，以及在添加工具、启动辅助程序或开启新会话时保留缓存复用。保持共享前缀稳定，并随着工作流推进追加新信息。对于不断增长的对话，使用隐式缓存；对于共享内容，使用显式断点。若两者都需要，可将它们结合使用；若不断变化的后缀不太可能被复用，则选择仅显式缓存模式。比较缓存读取、写入、输入总成本和延迟，找出最适合你的代理的方案。要应用这些模式，请参阅适用于 OpenAI 模型的 Amazon Bedrock 提示缓存文档，并查看各 AWS 区域和端点的模型可用情况。从一个工作流开始，随着对话增长，衡量节省效果。In this post, we covered how to cache shared tools and instructions, grow the cache across agent turns, and preserve reuse when adding tools, launching helpers, or starting new sessions. Keep the shared prefix stable and append new information as the workflow progresses. Use implicit caching for growing conversations and explicit breakpoints for shared content. Combine them when you need both, or choose explicit-only mode when a changing suffix is unlikely to be reused. Compare cache reads, writes, total input cost, and latency to find what works best for your agent. To apply these patterns, follow the Amazon Bedrock prompt caching documentation for OpenAI models and check model availability by AWS Region and endpoint. Start with one workflow and measure the savings as your conversations grow.

关于作者About the author

克劳迪奥·马佐尼Claudio Mazzoni

Claudio 是 Amazon Bedrock 市场推广团队的高级解决方案架构师。Claudio 擅长引导客户踏上生成式 AI 之旅。工作之余，Claudio 喜欢与家人共度时光，也喜欢打理自家的花园。Claudio is a Sr Specialist Solutions Architect on the Amazon Bedrock GTM team. Claudio excels at guiding customers through their Gen AI journey. Outside of work, Claudio enjoys spending time with family and working in his home garden.