Skip to content

Refactors Prompt Caching#253

Merged
philipph-askui merged 10 commits intomainfrom
feat/prompt_caching
Apr 1, 2026
Merged

Refactors Prompt Caching#253
philipph-askui merged 10 commits intomainfrom
feat/prompt_caching

Conversation

@philipph-askui
Copy link
Copy Markdown
Contributor

@philipph-askui philipph-askui commented Mar 30, 2026

Note: The updated lock file bloats the size of this PR, as +1.933 -2.280 lines are changed there. Hence, the actual PR is only +739 - 113)

CAUTION: prompt caching will yet only work with the AnthropicVlmProvider!

Here is a minimal example to try it:

import logging

from askui import ComputerAgent
from askui.agent_settings import AgentSettings
from askui.model_providers.anthropic_vlm_provider import AnthropicVlmProvider
from askui.models.shared.agent_message_param import CacheControlEphemeralParam
from askui.models.shared.settings import ActSettings, MessageSettings
from askui.reporting import SimpleHtmlReporter

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)


# test and implement anthropic's automatic prompt caching, as explained here:
# https://platform.claude.com/docs/en/build-with-claude/prompt-caching#automatic-caching


def main() -> None:
    agent = ComputerAgent(
        display=1,
        reporters=[SimpleHtmlReporter()],
        settings=AgentSettings(vlm_provider=AnthropicVlmProvider()),
    )
    agent.act_settings.messages.provider_options = {
        "cache_control": CacheControlEphemeralParam()
    }
    with agent:
        agent.act("Open a new Chrome browser")

        agent.act("Navigate to www.askui.com")


if __name__ == "__main__":
    main()

@philipph-askui philipph-askui mentioned this pull request Mar 30, 2026
@mlikasam-askui
Copy link
Copy Markdown
Contributor

@philipph-askui Regarding the example code in the PR, it overrides the act_settings for a single act command and removes the system prompt. I would suggest documenting how to modify only the caching setting by accessing agent.act_settings.messages... instead.

Additionally, I recommend enabling caching by default and documenting how to disable this behavior, even if disabling it is generally not advisable.

@philipph-askui
Copy link
Copy Markdown
Contributor Author

philipph-askui commented Mar 31, 2026

@philipph-askui Regarding the example code in the PR, it overrides the act_settings for a single act command and removes the system prompt. I would suggest documenting how to modify only the caching setting by accessing agent.act_settings.messages... instead.

Additionally, I recommend enabling caching by default and documenting how to disable this behavior, even if disabling it is generally not advisable.

I updated the example accordingly.
However, we cannot enable prompt caching by default, as it will is not yet supported by all providers (e.g. bedrock or vertexAI are still missing)

@programminx-askui
Copy link
Copy Markdown
Collaborator

programminx-askui commented Mar 31, 2026

@mlikasam-askui,

I will not review this PR. If you give a go @philipph-askui will merge it.

@philipph-askui philipph-askui merged commit a5318c8 into main Apr 1, 2026
1 check passed
@philipph-askui philipph-askui deleted the feat/prompt_caching branch April 1, 2026 14:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants