Client-side media: Generate a local poster during video upload by adamsilverstein · Pull Request #79406 · WordPress/gutenberg

adamsilverstein · 2026-06-22T16:29:19Z

What

Generates a poster image from a video's first frame, client-side, during upload, and applies it to the Video block automatically.

Closes #79366. Part of #76756.

Why

When a video is uploaded there is no poster to show until the video itself loads, so the area can render blank/late, hurting perceived performance and layout stability. Producing a poster from the first frame at upload time lets the page render meaningful content immediately, and the poster is serialized into the block so it persists.

The original issue assumed we could reuse the poster step from the GIF-to-video work (#76942 / #78410). In practice that step runs vips on a GIF's first frame (a still-image decode) and cannot decode a real video, so there was no reusable video-frame extractor to build on. This PR adds a small, dependency-free extractor instead (the approach proven in swissspidy/media-experiments), and is implemented independently on trunk.

How

In @wordpress/upload-media:

New poster-utils.ts with getPosterFromVideo() — loads the video, seeks slightly past the start (to avoid an empty first frame), draws it to an OffscreenCanvas, and returns a JPEG File. No new dependencies.
Two new pipeline operations, AddPoster and UploadPoster. For video uploads, prepareItem queues AddPoster -> Upload -> UploadPoster:
- addPosterForItem generates the poster (best-effort: if the browser can't decode a frame, the video uploads with no poster).
- uploadPosterForItem uploads the poster as its own image attachment and pushes its URL to the block via onChange, setting the block's poster attribute. The poster upload runs independently so the video item completes immediately.

In @wordpress/block-library, the Video block's onSelectVideo now reads media.poster (set by the upload pipeline) in addition to the media-library media.image.src.

Graceful fallback

Frame extraction is fully guarded; an unplayable codec, a seek timeout, or a missing canvas context results in no poster and a normal upload.

Note on `featured_media`

Linking the poster as the video attachment's featured_media (so it is also exposed via the REST API / media library) is intentionally left as a follow-up: @wordpress/upload-media delegates all server I/O to injected settings callbacks and has no direct REST dependency, and the poster already persists via the block's poster attribute in post content. Happy to add it here if reviewers prefer, via a new settings callback.

Testing Instructions

Build the editor and open a new post with client-side media processing active.
Add a Video block and upload an MP4/WebM (drag-and-drop or the upload button).
Confirm a poster image is generated and applied: the block gains a poster attribute and <video poster="..."> renders it.
Reload the post and confirm the poster persists (serialized in the block).
Edge: upload a video the browser cannot decode and confirm it still uploads, just without a poster.

Automated tests

Unit: npx wp-scripts test-unit-js packages/upload-media/src/store/test/private-actions.js — covers the video op sequence and the poster handlers (success, graceful failure, no-op).
E2E: client-side-media-processing.spec.js adds a test that generates a short WebM at runtime via MediaRecorder (so no binary fixture is committed), uploads it, and asserts the block gets a real poster image URL.

Add poster-utils.ts with getPosterFromVideo(), which extracts the first usable still frame of a video client-side using HTMLVideoElement and an OffscreenCanvas. Seeks slightly past the start to avoid an empty frame and falls back to JPEG where WebP is unsupported. No new dependencies.

Add AddPoster and UploadPoster operations to the upload pipeline. For video uploads, prepareItem now queues AddPoster -> Upload -> UploadPoster: - addPosterForItem generates a poster from the video's first frame (best-effort; on failure the video uploads without a poster). - uploadPosterForItem uploads the poster as its own image attachment and pushes its URL to the video block via onChange, setting the block's poster attribute. The poster upload runs independently so the video item completes immediately. The Video block's onSelectVideo now reads media.poster (set by the upload pipeline) in addition to the media-library media.image.src. Fixes #79366.

Unit tests cover the prepareItem video operation sequence and the addPosterForItem/uploadPosterForItem handlers (success, graceful failure, and no-op paths). The e2e test generates a short WebM at runtime via MediaRecorder (so no binary fixture of uncertain provenance is committed), uploads it into a Video block, and asserts the block gains a real poster image URL.

github-actions · 2026-06-22T16:29:52Z

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Co-authored-by: adamsilverstein <adamsilverstein@git.wordpress.org>
Co-authored-by: swissspidy <swissspidy@git.wordpress.org>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

github-actions · 2026-06-22T16:38:26Z

Size Change: +622 B (+0.01%)

Total Size: 7.92 MB

📦 View Changed

Filename	Size	Change
`build/scripts/block-library/index.min.js`	369 kB	+8 B (0%)
`build/scripts/upload-media/index.min.js`	16.9 kB	+614 B (+3.77%)

_{compressed-size-action}

…eration # Conflicts: # packages/upload-media/src/store/private-actions.ts # packages/upload-media/src/store/test/private-actions.js # test/e2e/specs/editor/various/client-side-media-processing.spec.js

github-actions · 2026-08-12T16:02:14Z

Flaky tests detected in fde9b21.
Some tests passed with failed attempts. The failures may not be related to this commit but are still reported for visibility. See the documentation for more information.

🔍 Workflow run URL: https://github.com/WordPress/gutenberg/actions/runs/31613185208
📝 Reported tests:

can use focal point picker to set the focal point of the cover image in /test/e2e/specs/editor/blocks/cover.spec.js, passed after 1 failed attempt.

Error: expect(locator).toHaveValue(expected) failed

Locator:  getByRole('spinbutton', { name: 'Focal point left position' })
Expected: "20"
Received: "50"
Timeout:  5000ms

Call log:
  - Expect "toHaveValue" with timeout 5000ms
  - waiting for getByRole('spinbutton', { name: 'Focal point left position' })
    14 × locator resolved to <input min="0" step="1" max="100" value="50" type="number" autocomplete="off" inputmode="numeric" id="inspector-input-control-2" aria-label="Focal point left position" class="components-input-control__input deb-b-b-bad-dff-1tbg82s em5sgkm3"/>
       - unexpected value "50"

    at /home/runner/work/gutenberg/gutenberg/test/e2e/specs/editor/blocks/cover.spec.js:417:34

# Conflicts: # packages/upload-media/src/store/private-actions.ts # packages/upload-media/src/store/test/private-actions.js # test/e2e/specs/editor/various/client-side-media-processing.spec.js

github-actions · 2026-09-15T06:03:56Z

🤖 PR meta 🤖

📦 Bundle size

Size Change: +629 B (+0.01%)

Total Size: 8.21 MB

📦 View Changed

Filename	Size	Change
`build/scripts/block-library/index.min.js`	364 kB	+7 B (0%)
`build/scripts/upload-media/index.min.js`	17.1 kB	+622 B (+3.78%)

_{9db3645 Run}

⚡ Performance

Show the results

Client side metrics exclude the server response time.

front-end-block-theme

Metric	`68abe7a`	trunk	% Change
timeToFirstByte	61.8 ms +8.9% -5.18%	59.55 ms +3.27% -3.44%	3.78%
largestContentfulPaint	104 ms +9.62% -3.85%	96 ms +4.17% -2.08%	8.33%
lcpMinusTtfb	45 ms +6.56% -20.78%	35.6 ms +8.57% -3.09%	26.4%
wpBeforeTemplate	30.17 ms +15.15% -3.25%	29.16 ms +2.64% -1.41%	3.46%
wpTemplate	26.55 ms +10.51% -5.5%	26.05 ms +3.03% -5.26%	1.92%
wpTotal	57.88 ms +8.9% -5.29%	55.83 ms +3.33% -3.71%	3.67%
wpMemoryUsage	7.53 MB +0% -0%	7.49 MB +0% -0%	0.46%
wpDbQueries	17 +0% -0%	17 +0% -0%	0%

front-end-classic-theme

Metric	`68abe7a`	trunk	% Change
timeToFirstByte	53.85 ms +4.92% -4.83%	44.65 ms +5.6% -2.69%	20.6%
largestContentfulPaint	112 ms +3.57% -3.57%	96 ms +0% -4.17%	16.67%
lcpMinusTtfb	57.85 ms +3.98% -1.99%	48.85 ms +1.94% -1.23%	18.42%
wpBeforeTemplate	28.85 ms +7.73% -3.43%	27.15 ms +6.7% -2.28%	6.26%
wpTemplate	20.58 ms +6.9% -3.06%	14.57 ms +4.67% -2.81%	41.25%
wpTotal	50.59 ms +4.27% -5.04%	41.71 ms +5.3% -2.4%	21.29%
wpMemoryUsage	6.15 MB +0% -0%	5.97 MB +0% -0%	2.89%
wpDbQueries	14 +0% -0%	10 +0% -0%	40%

media-processing

Metric	`68abe7a`	trunk	% Change
mediaProcessingJpeg	406.41 ms +0.68% -1.01%	403.97 ms +1.32% -1.28%	0.6%
mediaProcessingAvif	6074.87 ms +0.23% -0.15%	6108.47 ms +0.4% -0.19%	-0.55%
mediaProcessingJpegToAvif	4209.65 ms +0.44% -0.3%	4221.9 ms +0.71% -0.41%	-0.29%

media-upload

Metric	`68abe7a`	trunk	% Change
jpegUploadProcessing	1722.23 ms +15.82% -17.24%	1440.34 ms +1.05% -0.39%	19.57%
pngUploadProcessing	225.54 ms +1.02% -4.86%	223.8 ms +6.05% -3.84%	0.78%
largeJpegUploadProcessing	1431.71 ms +0.19% -0.71%	1414.82 ms +0.37% -0.42%	1.19%
multipleImageUploadProcessing	1673.22 ms +29.43% -1.86%	1587.9 ms +8.28% -0.11%	5.37%

post-editor

Metric	`68abe7a`	trunk	% Change
serverResponse	438.76 ms +1.33% -7.33%	425.31 ms +4.73% -7.91%	3.16%
firstPaint	215.4 ms +34.67% -19.61%	197.89 ms +2.32% -13.61%	8.85%
domContentLoaded	1157.89 ms +0.44% -2.29%	1118.9 ms +0.78% -1.42%	3.48%
loaded	1159.43 ms +0.42% -2.28%	1120.28 ms +0.79% -1.43%	3.49%
firstContentfulPaint	473.59 ms +3.87% -11.22%	455.11 ms +4.2% -3.03%	4.06%
firstBlock	3341.24 ms +0.89% -1.61%	3284.06 ms +0.97% -0.77%	1.74%
type	23.63 ms +3.34% -1.52%	22.17 ms +4.78% -5.14%	6.59%
typeWithoutInspector	23.34 ms +3.77% -3.3%	22.03 ms +2.63% -3.45%	5.95%
typeWithTopToolbar	31.26 ms +5.89% -5.31%	28.91 ms +8.99% -8.13%	8.13%
typeContainer	9.81 ms +6.93% -7.65%	9.61 ms +16.65% -3.43%	2.08%
focus	89.41 ms +6.78% -10.36%	82.86 ms +9.85% -5.93%	7.9%
firstFocus	233 ms +0% -0%	220.51 ms +0% -0%	5.66%
selectAll	518.46 ms +2.44% -2.14%	527.84 ms +3.96% -4.06%	-1.78%
listViewOpen	76.2 ms +7.23% -11.27%	74.44 ms +6.69% -4.58%	2.36%
inserterOpen	25.37 ms +3.94% -2.01%	26.29 ms +5.93% -6.96%	-3.5%
inserterHover	3.34 ms +14.67% -8.68%	3.25 ms +19.08% -5.54%	2.77%
inserterSearch	8.91 ms +4.04% -2.02%	9.5 ms +2.84% -8.63%	-6.21%
loadPatterns	672.58 ms +1.52% -1.96%	699.75 ms +4.93% -6.24%	-3.88%
wpTotal	426.91 ms +1.23% -7.43%	412.54 ms +5.15% -7.81%	3.48%
wpMemoryUsage	13.07 MB +0% -0%	13.03 MB +0% -0%	0.28%
wpDbQueries	54 +0% -0%	54 +0% -1.85%	0%

site-editor

Metric	`68abe7a`	trunk	% Change
serverResponse	339.67 ms +2.85% -3.45%	328.25 ms +5.66% -6.46%	3.48%
firstPaint	185.44 ms +27.05% -16.26%	186.98 ms +12.65% -23.49%	-0.82%
domContentLoaded	850.2 ms +1.07% -0.25%	847.32 ms +0.43% -0.99%	0.34%
loaded	850.99 ms +1.06% -0.25%	848.14 ms +0.44% -1%	0.34%
firstContentfulPaint	354.26 ms +6.27% -1.61%	341.01 ms +3.26% -1.38%	3.89%
firstBlock	3068.04 ms +0.99% -1.68%	3019.94 ms +0.62% -0.81%	1.59%
type	17.18 ms +0.87% -2.56%	16.83 ms +8.91% -5.64%	2.08%
navigate	112.5 ms +4.67% -3.93%	105.12 ms +2.05% -3.51%	7.02%
loadPatterns	1069.62 ms +16.38% -9.11%	1138.14 ms +8.57% -8.87%	-6.02%
loadPages	1095.28 ms +1.56% -1.75%	1095.91 ms +1.75% -1.69%	-0.06%
wpTotal	331.44 ms +2.9% -3.69%	319.94 ms +5.72% -6.67%	3.59%
wpMemoryUsage	12.02 MB +0% -0%	11.97 MB +0% -0%	0.41%
wpDbQueries	43.5 +1.15% -1.15%	44 +0% -2.27%	-1.14%

_{9db3645 Run}

adamsilverstein added 3 commits June 22, 2026 09:28

adamsilverstein requested review from ajitbohra and fabiankaegy as code owners June 22, 2026 16:29

Upload Media: add changelog entry for video poster generation

6cc2709

github-actions Bot added the [Package] Block library /packages/block-library label Jun 22, 2026

This was referenced Jun 22, 2026

Local poster generation during video upload #79366

Open

Client Side Media iteration for WordPress 7.1: stabilization, docs, and Core backports #76756

Closed

Merge remote-tracking branch 'origin/trunk' into add/video-poster-gen…

fde9b21

…eration # Conflicts: # packages/upload-media/src/store/private-actions.ts # packages/upload-media/src/store/test/private-actions.js # test/e2e/specs/editor/various/client-side-media-processing.spec.js

t-hamano added [Type] Enhancement A suggestion for improvement. [Feature] Client Side Media Media processing in the browser with WASM labels Aug 13, 2026

t-hamano added this to WordPress 7.2 Editor Tasks Aug 13, 2026

github-project-automation Bot moved this to 🔎 Needs Review in WordPress 7.2 Editor Tasks Aug 13, 2026

adamsilverstein and others added 3 commits August 13, 2026 13:39

Merge branch 'trunk' into add/video-poster-generation

eb834d2

Merge branch 'trunk' into add/video-poster-generation

631d5eb

Merge remote-tracking branch 'origin/trunk' into HEAD

d695b9f

# Conflicts: # packages/upload-media/src/store/private-actions.ts # packages/upload-media/src/store/test/private-actions.js # test/e2e/specs/editor/various/client-side-media-processing.spec.js

Merge branch 'trunk' into add/video-poster-generation

9db3645

adamsilverstein mentioned this pull request Sep 30, 2026

Client Side Media iteration for WordPress 7.2 and beyond #80159

Open

Navigation Menu

Repository navigation

Client-side media: Generate a local poster during video upload - #79406

Client-side media: Generate a local poster during video upload#79406
adamsilverstein wants to merge 9 commits into
trunkfrom
add/video-poster-generation

adamsilverstein commented Jun 22, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Jun 22, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Jun 22, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Aug 12, 2026

Uh oh!

github-actions Bot commented Sep 15, 2026 •

edited

Loading

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

adamsilverstein commented Jun 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Why

How

Graceful fallback

Note on featured_media

Testing Instructions

Automated tests

Uh oh!

github-actions Bot commented Jun 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Aug 12, 2026

Uh oh!

github-actions Bot commented Sep 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🤖 PR meta 🤖

📦 Bundle size

⚡ Performance

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

adamsilverstein commented Jun 22, 2026 •

edited

Loading

Note on `featured_media`

github-actions Bot commented Jun 22, 2026 •

edited

Loading

github-actions Bot commented Jun 22, 2026 •

edited

Loading

github-actions Bot commented Sep 15, 2026 •

edited

Loading