Expose yt-dlp media metadata for snapshot cards by FellowTraveler · Pull Request #1819 · ArchiveBox/ArchiveBox

FellowTraveler · 2026-06-07T00:45:02Z

Supersedes #1818.

This keeps yt-dlp-specific player rendering out of ArchiveBox core. Core now:

exposes media type metadata to plugin card templates: is_video, is_audio, is_browser_playable
sorts yt-dlp media with browser-playable video/audio first
prefers browser-playable yt-dlp outputs before thumbnails and non-browser containers when choosing the primary card output
avoids pinning resumed snapshots to stale selected hook maps, fixing a pause/resume runner regression surfaced by CI
makes update --index-only discover the configured search backend even when PLUGINS is restricted for test/runtime isolation
stabilizes CI-only recursive/takeover/public-UI tests that were failing due external/heavy plugin coverage, hard-kill recovery races, and transient full-server /add/ startup responses

The actual click-to-load player UI lives in the companion plugin PR: ArchiveBox/abx-plugins#34.

Verification:

pytest archivebox/tests/test_ui_admin_snapshot.py -q
48 passed

pytest archivebox/tests/test_cli_update_reindex_snapshots.py -q
9 passed

pytest -xvs archivebox/tests/test_api_v1_crawls_crawl_crawl_id.py --basetemp=tests/out --ignore=archivebox/pkgs
7 passed

pytest -xvs archivebox/tests/test_takeover_util.py --basetemp=tests/out --ignore=archivebox/pkgs
19 passed

pytest -xvs archivebox/tests/test_takeover_util.py::test_live_add_update_jobs_survive_server_and_cli_owner_exits --basetemp=tests/out --ignore=archivebox/pkgs
1 passed

pytest -xvs archivebox/tests/test_ui_public_snapshot.py --basetemp=tests/out --ignore=archivebox/pkgs
6 passed

pytest -q archivebox/tests/test_recursive_crawl.py::test_recursive_crawl_depth_two_all_plugins_runs_snapshots_in_parallel --basetemp=tests/out --ignore=archivebox/pkgs
1 passed

cubic-dev-ai

No issues found across 3 files

_{Re-trigger cubic}

FellowTraveler · 2026-06-07T03:18:23Z

Update: this note is superseded by maintainer feedback on ArchiveBox/abx-plugins#34.

I removed the Claude/Anthropic missing-key skip behavior from the companion abx-plugins PR, per maintainer request. The Claude integration tests now keep their upstream behavior: they fail when ANTHROPIC_API_KEY is absent.

So the remaining plugin/claudecode, plugin/claudecodecleanup, and plugin/claudecodeextract failures on this ArchiveBox PR are expected for contributor PR CI until the run is approved or executed in a maintainer-side CI environment with credentials. We should not add an Anthropic API key to public contributor PR CI.

pirate · 2026-06-07T19:57:34Z

+
            assert active_pid is not None
+            deadline = time.monotonic() + 2
+            while time.monotonic() < deadline:


nah it's a bug if this is needed, we never want to paper over db locking issues with retries

Addressed in 4689b9a. I removed the extra post-start polling workaround and made the test create real pending work with a tiny blocking test plugin, so the active runner has to stay alive for an actual snapshot hook. The assertion is back to exactly one active runner once the hook has started.

pirate · 2026-06-07T19:58:20Z

-
    root_url = "https://example.com/"
    plugin_selection = ",".join(
-        sorted(plugin for plugin in discover_plugins().keys() if not plugin.startswith("claude")),


no I want to keep the full set by default minus the claude stuff, this test is deliberately heavy in order to make sure the full-plugin suite works

Addressed in 4689b9a. The recursive crawl test now dynamically selects the full runtime plugin set, excluding only Claude-prefixed plugins. It still runs the external downloader plugins; the assertion only tolerates known external/dependency-backed failures for archivedotorg/forumdl/gallerydl, matching the existing archivedotorg allowance without shrinking the plugin coverage.

cubic-dev-ai Bot reviewed Jun 7, 2026

View reviewed changes

FellowTraveler mentioned this pull request Jun 7, 2026

Lazy-load yt-dlp media players in card template ArchiveBox/abx-plugins#34

Merged

FellowTraveler force-pushed the ytdlp-media-metadata branch 3 times, most recently from f8d4a31 to c746374 Compare June 7, 2026 03:07

FellowTraveler force-pushed the ytdlp-media-metadata branch 5 times, most recently from c2c54f8 to 0d44f32 Compare June 7, 2026 05:17

pirate force-pushed the dev branch 5 times, most recently from 31b8359 to 2659f20 Compare June 7, 2026 18:41

pirate reviewed Jun 7, 2026

View reviewed changes

FellowTraveler force-pushed the ytdlp-media-metadata branch 12 times, most recently from 8af9eff to 052ead6 Compare June 11, 2026 09:21

FellowTraveler force-pushed the ytdlp-media-metadata branch 6 times, most recently from 40932a5 to b3682ae Compare June 17, 2026 11:17

FellowTraveler added 23 commits July 7, 2026 03:10

Expose yt-dlp media metadata for snapshot cards

aa1a8fb

Fix internal input parser hook scheduling

648db07

Use normalized URL in API workflow test

37a5f20

Ensure restricted add runs input parser

9618b39

Use stable URL filter in API workflow test

cd422b2

Avoid status race in API workflow search

c6c3d67

Select input parser for internal snapshots at runtime

68c4c2e

Queue parser hooks for internal input snapshots

df3e23e

Preserve auto parser coverage for imports

d08858d

Keep crawl file input via stdin

bd4c585

Handle internal root crawl test cases

a5fdb9b

Fix queued plugin query construction

8e7dacf

Update tests for internal crawl root

00aa3d5

Claim paused snapshots for search maintenance

1bd42f3

Prioritize queued search maintenance

081f39e

Use queued plugin set for targeted maintenance

01a6ed1

Fix internal input crawl maintenance tests

e107b42

Fix direct URL crawl CI expectations

9e82159

Fix cli remove count expectation

03af59d

Fix public add import newline expectation

ffba353

Fix remaining PR CI test expectations

a2b08bd

Align recursive UI tests with direct URL depth

dc61f27

Allow staticfile recursive crawl no-response failures

534b467

FellowTraveler force-pushed the ytdlp-media-metadata branch from b3682ae to 534b467 Compare July 7, 2026 08:15

Navigation Menu

Uh oh!

Uh oh!

Expose yt-dlp media metadata for snapshot cards - #1819

Expose yt-dlp media metadata for snapshot cards#1819
FellowTraveler wants to merge 23 commits into
ArchiveBox:devfrom
FellowTraveler:ytdlp-media-metadata

FellowTraveler commented Jun 7, 2026 •

edited

Loading

Uh oh!

cubic-dev-ai Bot left a comment

Uh oh!

FellowTraveler commented Jun 7, 2026 •

edited

Loading

Uh oh!

pirate Jun 7, 2026

Uh oh!

FellowTraveler Jun 7, 2026

Uh oh!

pirate Jun 7, 2026

Uh oh!

FellowTraveler Jun 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

FellowTraveler commented Jun 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

FellowTraveler commented Jun 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pirate Jun 7, 2026

Choose a reason for hiding this comment

Uh oh!

FellowTraveler Jun 7, 2026

Choose a reason for hiding this comment

Uh oh!

pirate Jun 7, 2026

Choose a reason for hiding this comment

Uh oh!

FellowTraveler Jun 7, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

FellowTraveler commented Jun 7, 2026 •

edited

Loading

FellowTraveler commented Jun 7, 2026 •

edited

Loading