refetch

When Claude Code's WebFetch gets blocked, refetch reads the page.

A Claude Code plugin for the pages WebFetch cannot read: HTTP 403 and 429, "Just a moment..." challenges, JavaScript-only pages and timeouts. You don't change how you work. When a read fails, refetch steps in on its own and Claude gets the page.

> What does the headline on https://www.indeed.com/ say?

  WebFetch      https://www.indeed.com/  ->  HTTP 403 Forbidden
  refetch hook  tells Claude to call fetch_page with this URL
  fetch_page    https://www.indeed.com/  ->  HTTP 200, Chrome-like client

  The headline reads "Your next job starts here".

In a test of 10 everyday prompts, mostly on sites that block bots, Claude read 2 pages with WebFetch alone and 6 with refetch. That was before the headless Chrome step, which adds JavaScript-only pages and some challenges.

/plugin install refetch --marketplace samgozman/refetch

• Automatic. Hooks catch the failed read and point Claude at refetch in the same turn. No prompt changes, no new habits.
• Fast and small. One Go binary of about 8 MB to download. The first step takes about 1 s. Headless Chrome runs only when a page needs it, and uses the Chrome you already have.
• Gets past what stops WebFetch. A client with Chrome's TLS fingerprint first, then real headless Chrome for challenges and JavaScript-only pages.
• Plays well with others. Works next to context-mode and RTK, and never denies or rewrites a tool call.
• Shows its value. refetch stats reports how many reads failed, how many refetch picked up and which step resolved them. All data stays on your machine.
• Safe by default. Public addresses only, one page per call, no crawling, no cookies kept.

refetch runs on macOS and Linux (arm64 and amd64). On Windows, run Claude Code inside WSL, where refetch uses its Linux build; install Chrome or Chromium inside WSL for the headless Chrome step. Native Windows is not supported yet: the plugin's launcher is a shell script, and the headless step stops Chrome with Unix process groups.

In Claude Code 2.1.275 or later:

/plugin install refetch --marketplace samgozman/refetch

On older versions, use two commands:

/plugin marketplace add samgozman/refetch
/plugin install refetch@refetch

The first session after install downloads the refetch binary for your platform (about 8 MB) from this repo's GitHub release and checks it against the sha256 pinned in the plugin.

Without this rule, every rescue waits on a permission prompt and fails outright under claude -p. Add it to ~/.claude/settings.json:

{
  "permissions": {
    "allow": ["mcp__plugin_refetch_refetch__fetch_page"]
  }
}

Claude Code does not auto-update plugins from third-party marketplaces unless you turn it on: /plugin → Marketplaces → refetch → Enable auto-update. New versions load on the next launch or /reload-plugins. To update by hand, run claude plugin update refetch@refetch.

Ask Claude to run refetch doctor, or run it yourself. It checks the binary and plugin versions, the permission rule, Chrome, the stats file and other hooks on WebFetch.

refetch sits after Claude Code's own web reads. It changes nothing while they work and steps in only when one fails.

flowchart TD
    Claude["Claude reads a URL"] --> WebFetch["WebFetch"]
    WebFetch -. "with context-mode:<br/>WebFetch is denied" .-> Ctx["ctx_fetch_and_index"]
    WebFetch --> Read{"Got the page?"}
    Ctx --> Read
    Read -- "yes" --> Done(["Claude answers from the page"])
    Read -- "no: error, HTTP 403/429/5xx,<br/>challenge or empty page" --> Hook["refetch hook adds a note:<br/>call fetch_page for this URL"]
    Hook --> Client

    subgraph FetchPage ["fetch_page, refetch's MCP tool"]
        Client["1. Chrome-like client<br/>Chrome TLS fingerprint, about 1 s"]
        Client -- "challenge or<br/>JavaScript-only page" --> Headless["2. Headless Chrome<br/>your installed Chrome, 15 s at most"]
        Client -- "page" --> Text["Page as Markdown"]
        Headless -- "page" --> Text
        Client -- "HTTP 403/429<br/>or error" --> Handoff["Hand-off: why it failed"]
        Headless -- "still blocked" --> Handoff
    end

    Text --> Rescued(["Claude answers from the page"])
    Handoff --> Other(["Claude tries a browser tool<br/>or other sources"])

The hook and fetch_page record every step locally, so refetch stats can show how often each path was taken. Step by step:

1. WebFetch runs as usual. refetch only reacts after a failed read, or after a "successful" read that is really an error page, challenge or empty JavaScript shell.
2. The hook adds a note next to the failed result telling Claude to call fetch_page with that URL.
3. fetch_page reads the page with a Go HTTP client whose TLS handshake looks like Chrome's (about 1 s, no JavaScript). A plain Go client is only its backup for the Chrome-like client's own TLS or protocol errors.
4. For a challenge or JavaScript-only page, it runs your installed Chrome or Chromium headless, for 15 s at most. A challenge counts even when it comes as a 403, as Cloudflare's does (cf-mitigated: challenge). A plain 403 or 429 skips this step, and so does a machine without Chrome.
5. If both steps fail, the result says why and suggests a browser tool if you have one.

fetch_page returns a page as Markdown: headings, lists, links, tables and code blocks kept, scripts and hidden elements dropped. Links are absolute, and search engines' click-through links point at their targets, so Claude can follow a search result or a "next page" link. Only http, https and mailto links keep their target, and tracking parameters (utm_*, fbclid, gclid) are dropped. Pass format: "text" for plain text instead. A PDF, an image or another file gets a clear error instead of its bytes.

Claude can pass render: true to read a page with headless Chrome even when the first step got it (for content that scripts fill in), render: false to skip headless Chrome for one call, and save: true to get a file path and a short preview instead of the whole page in context (the file is .md, or .txt with format: "text"). If Chrome cannot read a page the first step got, the result is the first step's text, with a note.

context-mode denies every WebFetch and sends Claude to ctx_fetch_and_index. refetch also watches that tool's failures and challenge previews, and counts the deny as a redirect, not a failure. refetch never denies or rewrites a tool call, so it does not conflict with context-mode or RTK.

$ refetch stats --since 30d

The report shows how many web reads (WebFetch and ctx_fetch_and_index) failed, how many refetch picked up and how many it resolved, split by which step won. It also shows how often the headless Chrome step ran, timed out or was skipped, and why. Other flags are --by host, --session <id> and --json. refetch stats --reset deletes everything.

The data stays on your machine in ~/.local/share/refetch/events.db ($XDG_DATA_HOME or REFETCH_DATA move it). URLs are stored with the query string and fragment removed. Rows older than 180 days are rolled up into daily totals. Updates and uninstalls keep the file.

Put settings in ~/.config/refetch/config.toml, or set the REFETCH_* environment variables, which override the file. REFETCH_CONFIG points to another file. Everything is on by default.

# ~/.config/refetch/config.toml
render = "off"
stats = false

Headless Chrome uses REFETCH_CHROME if set, then the standard macOS app paths, then google-chrome or chromium on PATH, then Playwright's Chromium (in PLAYWRIGHT_BROWSERS_PATH, /ms-playwright, /opt/pw-browsers or ~/.cache/ms-playwright). refetch never downloads a browser. Chrome's sandbox does not run as root, so when refetch runs as root (in a container, for example), it runs Chrome as the nobody user on Linux and skips the step elsewhere. Chrome's sandbox needs user namespaces, which Docker's default seccomp profile blocks, so in a container the step needs one that allows them (--security-opt seccomp=unconfined is the blunt way). refetch skips a Playwright Chromium that other users can write to, as in Playwright's Docker images; refetch doctor names it, and setting REFETCH_CHROME to it uses it anyway.

To turn refetch off entirely, disable the plugin in /plugin.

refetch reads pages with a Chrome TLS fingerprint, and reads challenge and JavaScript-only pages with your installed Chrome in headless mode. This gets it past some bot protection. You are responsible for following the terms of the sites you read with it.

These limits are always on, whatever the settings:

• Public addresses only, on every connection and redirect, headless Chrome included. No localhost, private networks, cloud metadata addresses or this machine's own addresses.
• One page per call. refetch follows no links and does no crawling. While headless Chrome renders a page, that page's own scripts run for up to render_timeout and can make requests to public hosts, as in any browser.
• No cookies are kept between calls: each render uses a throwaway Chrome profile.
• Pages are capped at 2 MB.

What the address check cannot know: an internal network that uses public IP addresses, or a service that trusts your IP address, looks public to it. refetch also ignores HTTP_PROXY and HTTPS_PROXY on purpose, so it does not work where all traffic must go through a corporate proxy. See SECURITY.md for what refetch defends against and what you trust when you install it.

robots.txt is not consulted by default, because refetch reads only the one URL your session was already sent to. Set respect_robots = true to consult it.

Every release archive carries a GitHub build provenance attestation. To check that a binary was built by this repo's release workflow:

gh attestation verify refetch_<version>_darwin_arm64.tar.gz --repo samgozman/refetch

The plugin's shim also checks each download, and the cached binary on every run, against the sha256 values pinned in the plugin.

/plugin uninstall refetch@refetch removes the hooks and the MCP server. The plugin writes nothing to your Claude Code settings. Downloaded binaries are in ~/.cache/refetch, and the stats are in ~/.local/share/refetch.

Requires Go 1.27.

make build                       # build/refetch
build/refetch get https://example.com
build/refetch doctor

To run a local build through the plugin, set REFETCH_BIN to its path before starting Claude Code. make test runs the tests, which never touch the network, and make lint runs the linters.

Bug reports, blocked sites and pull requests are welcome. See CONTRIBUTING.md for the setup and checks, and SECURITY.md to report a vulnerability privately.

MIT. refetch includes tls-client by Bogdan Finn (BSD-4-Clause): this product includes software developed by Bogdan Finn. Every release archive ships THIRD_PARTY_NOTICES with the license text of each dependency.