feat: add task API for bidirectional background worker communication#2319
Open
nicolas-grekas wants to merge 2 commits intophp:mainfrom
Open
feat: add task API for bidirectional background worker communication#2319nicolas-grekas wants to merge 2 commits intophp:mainfrom
nicolas-grekas wants to merge 2 commits intophp:mainfrom
Conversation
cf660f6 to
bb3febd
Compare
bb3febd to
beec9b4
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR builds on top of #2287 (background workers), which is the first commit.
The following description is only about what's currently the second commit.
An overview of both features combined together can be reviewed by reading the doc attached to this PR.
Summary
Adds a task API for bidirectional communication between HTTP workers and background workers. While
set_vars/get_varspushes config from background workers to HTTP workers, tasks enable the reverse: HTTP workers dispatch work to background workers and stream results back.PHP API
Sender-side:
frankenphp_worker_task_send(string $name, array $payload, float $timeout = 30.0): resource- sends a task to a named background worker, returns a readable stream for resultsfrankenphp_worker_task_read(resource $stream): ?array- reads the next update, returnsnullon clean completionReceiver-side:
frankenphp_worker_task_receive(): ?array- dequeues a pending task to process, returns[$stream, $payload]ornullfrankenphp_worker_task_update(resource $stream, array $data): void- sends a progress update or result back to the senderAll payloads and updates follow the same type constraints as
set_vars:null,bool,int,float,string,array(nested), and enums. Objects and resources are rejected.How it works
task_send('worker-name', $payload)- blocks until a background worker picks up the task"task\n"on the signaling stream, callstask_receive()to get[$stream, $payload]task_update($stream, $data)task_read($stream)- wakes up on each updatefclose($stream)when done - sender getsnullfromtask_read()fclose($stream)to acknowledge completionBlocking behavior
task_sendblocks until the background worker picks up the task (with timeout). Uses a buffered channel (size 1) so the sender can enqueue before the"task\n"signal reaches the receiver.task_readblocks until the next update arrives or the stream is closed. The task stream fromtask_sendisstream_select-compatible, so callers can check readability before callingtask_read.fclose()before the background worker completes. Cancelled tasks are detected attask_receive()time and skipped.Crash detection
fclose($stream)(crash,exit(), fatal error),task_read()throwsRuntimeExceptionfcloseby the background worker) returnsnullfromtask_read()EG(flags) & EG_FLAGS_IN_SHUTDOWNto distinguish explicitfclose()from resource cleanup during script exitPooling
Named background workers support
num > 1to run multiple threads. All threads share the same task channel - tasks are distributed automatically across the pool. The signaling stream fans out"task\n"to all threads.Example
Architecture
taskRequeststruct coordinates sender and receiver viaclosedSidesatomic counter - task slot is freed when both sides closetaskFIFOprovides bounded backpressure for updates (max 16 items, blocks on push when full)task_update, sender detects them viastream_selectbackgroundFdListstored onbackgroundWorkerStatefor O(1) lookup (no worker search)Test coverage
10 tests covering: basic task send/receive, progress streaming, non-worker sender, crash before receive, crash mid-task, cancellation, cancel-then-send recovery, cancel-then-crash, pooling (num=2).
Documentation
Task API section added to
docs/background-workers.md.