Skip to content

Latest commit

 

History

History
720 lines (593 loc) · 41.6 KB

README.md

File metadata and controls

720 lines (593 loc) · 41.6 KB

Select-String measurements

This suite separates the sequential search engine from the PowerShell adapter. SelectString.Engine.cs is compiled directly into the standalone xUnit and BenchmarkDotNet projects; neither references a PowerShell assembly. Keep that file independent of PowerShell. Public result types, parameter binding, path resolution, object conversion and pipeline output belong in SelectString.cs.

The first milestone preserves the shipped context algorithms and StreamReader semantics. Result text owns its storage. Input strings are records, including embedded newlines. File context resets for each file; pipeline context persists between records. File scanning calls a synchronous sink and stops when it returns false. -List/-Quiet wait for the necessary context and close the file before emitting. Concurrency, pooled decoding, compiled regex selection and specialized existence-only matching remain subsequent experiments.

The allocation phase separates matching metadata (MatchData, a value type) from result construction. A constrained struct factory lets scanning and both context trackers use caller-owned reference objects. Detailed output uses MatchInfo directly; the core does not reference that type. Raw and boolean Quiet output retain a lightweight representation. Logical context entries are immutable value types in the ring buffer, while matching results remain shared references. Filtered MatchInfo objects preserve shipped identity/cloning behavior, including downstream line mutations that affect later context. The factory owns context assignment; results and context arrays must not be reused after emission.

Performance module and entry scripts

SelectString.Performance.psd1 defines the module and its exports. Implementations, parameter definitions and full help live in SelectString.Performance.psm1. Measure-SelectString.ps1, New-EtwCapture.ps1, New-LogFixture.ps1, and test/SelectString/Compare-SelectString.ps1 load dynamic parameters from those definitions, force-reload through Initialize-Performance.ps1, and delegate. Existing command lines still work. The module owns build discovery, worker lifetime and directory completion classes; completion uses IArgumentCompleter and IArgumentCompleterFactory.

For full help in a fresh interactive session, initialize first (PowerShell does not load a dynamic script's help target just to display help):

./test/perf/SelectString/Initialize-Performance.ps1
Get-Help Invoke-SelectStringMeasurement -Full
Get-Help ./test/perf/SelectString/New-EtwCapture.ps1 -Full
Get-Help New-SelectStringLogFixture -Full
Get-Help Invoke-SelectStringComparison -Full

The initializer imports globally for interactive use. It is deliberately separate from the manifest's ScriptsToProcess: it imports the manifest itself. Ordinary script execution and tab completion initialize automatically. Avoid using module in wrappers so parse-time class references do not pin old definitions during edits.

Run the harness checks with a built executable:

./test/SelectString/Test-PerformanceScripts.ps1 -PwshPath ./tmp/SelectString/candidate/pwsh.exe

These check help, completion, changed-class reload, worker failures/timeouts, fixture validation and small real measurements. Capture orchestration uses a fake recorder; the checks do not start WPR. Artifacts stay under tmp/SelectString. Measurement JSON also records HostSupportsVirtualTerminal to help explain host-dependent highlight allocations.

Build and check compatibility

Run from the repository root. Use separate worktrees and output directories to avoid assembly reuse. The helper regenerates both resources and types in each tree, publishes a framework-dependent Release build, copies reference assemblies needed by Add-Type/Pester, and records build provenance.

git worktree add --detach ../SelectString-baseline 17f1446e5c4d88636e0d65bfedbaf8fe1bd5aaa6
git worktree add --detach ../SelectString-first-extraction 41b3169b4

./test/SelectString/Build-Comparison.ps1 -SourceRoot ../SelectString-baseline -OutputDirectory ./tmp/SelectString/baseline
./test/SelectString/Build-Comparison.ps1 -SourceRoot ../SelectString-first-extraction -OutputDirectory ./tmp/SelectString/first-extraction
./test/SelectString/Build-Comparison.ps1 -SourceRoot . -OutputDirectory ./tmp/SelectString/candidate

dotnet test test/SelectString/SelectString.Engine.Tests.csproj -c Release
./tmp/SelectString/candidate/pwsh.exe -NoProfile -File ./test/SelectString/Test-ParallelStop.ps1
./test/SelectString/Compare-SelectString.ps1 -Baseline ./tmp/SelectString/baseline/pwsh.exe -Candidate ./tmp/SelectString/candidate/pwsh.exe

& ./tmp/SelectString/candidate/pwsh.exe -NoProfile -Command {
    Import-Module Pester -RequiredVersion 4.10.1
    $result = Invoke-Pester -Script ./test/powershell/Modules/Microsoft.PowerShell.Utility/Select-String.Tests.ps1 -PassThru
    if ($result.FailedCount) { exit 1 }
}

On Unix use pwsh instead of pwsh.exe. The existing Pester file uses Pester 4 discovery semantics. The differential script does not require Pester. It compares public result APIs, output fields, captures, formatting, context, errors, warnings, FileInfo argument transformation, filtering identity and file closure. Snapshot files are written under test/SelectString/bin/comparison by default.

Standalone engine

dotnet run -c Release --project test/perf/SelectString/SelectString.Engine.Benchmarks.csproj -- --filter '*LargeFile*' --job short --artifacts test/perf/SelectString/bin/bdn
dotnet run -c Release --project test/perf/SelectString/SelectString.Engine.Benchmarks.csproj -- --filter '*PatternSetup*' --job short --artifacts test/perf/SelectString/bin/bdn-setup

Omit --job short for longer measurements. Filter by scenario name, or use '*' for the full matrix. Scanning and pattern preparation are separate benchmarks. File generation is outside the measured operation. The result sink counts matches without retaining them. MemoryDiagnoser records allocations and collections. The project pins a BenchmarkDotNet preview with .NET 11 support and keeps language properties local to avoid copying XML namespaces from the repository's older MSBuild props into generated benchmark projects.

For a controlled core-only comparison, SELECTSTRING_ENGINE_SOURCE can select the first extraction's engine while retaining the same benchmark code and data:

$savedEngine = $env:SELECTSTRING_ENGINE_SOURCE
try {
    $env:SELECTSTRING_ENGINE_SOURCE = (Resolve-Path ../SelectString-first-extraction/src/Microsoft.PowerShell.Commands.Utility/commands/utility/SelectString.Engine.cs).Path
    dotnet run -c Release --project test/perf/SelectString/SelectString.Engine.Benchmarks.csproj -- --filter '*NoMatchContext*False*' --job medium --artifacts test/perf/SelectString/bin/core-first
} finally {
    $env:SELECTSTRING_ENGINE_SOURCE = $savedEngine
}
dotnet run -c Release --project test/perf/SelectString/SelectString.Engine.Benchmarks.csproj -- --filter '*NoMatchContext*False*' --job medium --artifacts test/perf/SelectString/bin/core-current

Reader buffer experiment

ReaderBufferScanning compares fixed 1 KiB, 16 KiB and 64 KiB buffers in the complete engine file scan. It covers sparse captures and all matches on the shared 300 MiB log, 256 small files, and a first-line match with early termination. The FileStream configuration and regex options stay unchanged. It requires the current engine (unset SELECTSTRING_ENGINE_SOURCE).

./test/perf/SelectString/New-LogFixture.ps1  # only if the shared fixture is absent
$env:SELECTSTRING_LOG_FIXTURE = (Resolve-Path ./tmp/SelectString/log-fixture).Path
dotnet run -c Release --project test/perf/SelectString/SelectString.Engine.Benchmarks.csproj -- --filter '*ReaderBufferScanning*' --job short --affinity 1 --artifacts tmp/SelectString/buffers-bdn

Setup checks the manifest/hash and validates result order and named captures for each large-log buffer variant. File generation, validation and regex construction are outside timing. MemoryDiagnoser reports scan allocations; the sink counts without retaining results. These are warm-cache engine measurements, not shipping cmdlet comparisons. Confirm a promising choice end-to-end before adoption.

Larger buffers can decode beyond an early match and encounter invalid bytes that the default reader never reaches. A regression test demonstrates this with strict UTF-8 decoding. Buffer size therefore affects observable early-exit behavior as well as memory and throughput; an unconditional increase is not semantics-neutral.

Compiled-regex experiment

PatternFamilyScanning tests literal text, alpha.*omega, alpha.+omega, SQL durations, order IDs, image extensions, PR references, Markdown headings and secret assignments. PatternFamilies.cs records source URLs and authored inputs with expected spans and groups. Standalone tests cover both compilation/case settings and both AllMatches settings; benchmark setup validates its own selected variant. These catalog expressions are search workloads, not complete domain validators.

dotnet run -c Release --project test/perf/SelectString/SelectString.Engine.Benchmarks.csproj -- --filter '*PatternFamilyScanning*Records: 10000,*CaseSensitive: True,*Input: Mixed*' --job short --affinity 1 --artifacts tmp/SelectString/pattern-families

This selects 36 cases: nine families, interpreted/compiled, fresh/reused engines. Each operation matches 10,000 prepared records. Mixed cycles equally through a successful match, an absent prefix, a short near miss and a roughly 4 KiB near miss. This deliberately exercises failure paths; it is not a measured real-world input distribution. The enum also supports each input kind separately; Records accepts 1 or 10000 and CaseSensitive accepts both settings. Filter these dimensions to keep runs focused. Whole-line anchors are preserved. There is no file/pipeline overhead in these benchmarks; confirm promising cases end-to-end before adoption.

The focused controls in RESULTS-patterns.md can be repeated with:

$filters = foreach ($family in 'Literal','DotStar','DotPlus') {
    foreach ($inputKind in 'Match','Absent','LongNearMiss') {
        "*PatternFamilyScanning.ConstructAndMatch*Records: 10000,*CaseSensitive: True,*Family: $family, Input: $inputKind*"
    }
}
dotnet run -c Release --project test/perf/SelectString/SelectString.Engine.Benchmarks.csproj -- --filter $filters --job short --affinity 1 --artifacts tmp/SelectString/pattern-controls

End-to-end pipeline workloads are available through Measure-SelectString.ps1:

./test/perf/SelectString/Measure-SelectString.ps1 -PwshPath ./tmp/SelectString/candidate/pwsh.exe -Revision current -Scenario PipelineCaptures,PipelineStop,FileStop -RecordCount 10000 -MatchEvery 100 -WarmupIterations 60 -Iterations 25 -Artifacts ./tmp/SelectString/pipeline-current

PipelineCaptures feeds all records into one cmdlet invocation using named capture groups. RecordCount controls input size and MatchEvery controls match spacing; the first record matches. PipelineStop uses the same input and appends Select-Object -First 1. FileStop searches the ordinary file fixture and appends the same downstream command. Total time includes termination and cleanup; first result is timed in a separate instrumented pass. Fixtures and capture validation are outside timing. These options do not change the historical Records scenario. JSON records the pipeline parameters and validated result count.

Use -LatencyIterations 25 to repeat the separate instrumented counting pass. FirstResultMs is the median, and FirstResultSamplesMs retains every observation (including null for an invocation with no output). The default remains one call; -SkipCounting skips every latency call. Latency sampling follows throughput measurement and includes the counting sink's overhead, so it is not directly comparable with the null-redirection total time. Repetition exposes outliers but does not remove instrumentation bias. Result counts must agree across samples.

RegexScanning compares interpreted and compiled regexes with the same production reader-buffer policy. ConstructAndScan creates a fresh engine on every timed operation, including regex compilation and first-use JIT. ReuseAndScan uses an engine warmed during setup. Neither benchmark changes the cmdlet's regex policy. These measure engine work; cmdlet startup, binding and output costs are excluded.

The matrix includes sparse captures, all matches, three patterns, a literal regex, first-match termination, 256 small files, and a single string. File sizes range from 4 KiB to 300 MiB. String and small-file controls have fixed inputs: select SizeKiB 4 for those cases to avoid redundant runs. Setup generates deterministic logs and validates named captures outside timing; no existing fixture is needed. The early case models the shared engine stop-after-first path, not all the differences between the cmdlet's Quiet and List output modes.

dotnet run -c Release --project test/perf/SelectString/SelectString.Engine.Benchmarks.csproj -- --filter '*RegexScanning*SizeKiB: 4,*String*' '*RegexScanning*SizeKiB: 307200,*LogAllMatches*' '*RegexScanning*SizeKiB: 307200,*LogSparse*' --job short --affinity 1 --artifacts tmp/SelectString/regex-initial

Use the size sweep to investigate amortization before proposing a selection rule. Short runs are exploratory. Confirm any promising choice with repeated end-to-end shipping/current/experimental comparisons, including compilation in each timed cmdlet invocation and preserving first-result measurements.

The construction-inclusive sweep used in RESULTS-regex.md is:

dotnet run -c Release --project test/perf/SelectString/SelectString.Engine.Benchmarks.csproj -- --filter '*RegexScanning.ConstructAndScan*SizeKiB: 4,*' '*RegexScanning.ConstructAndScan*SizeKiB: 1024,*Log*' '*RegexScanning.ConstructAndScan*SizeKiB: 16384,*Log*' '*RegexScanning.ConstructAndScan*SizeKiB: 307200,*Early*' '*RegexScanning.ConstructAndScan*SizeKiB: 16384,*Multiple*' '*RegexScanning.ConstructAndScan*SizeKiB: 16384,*Literal*' --job short --affinity 1 --artifacts tmp/SelectString/regex-sweep

File-size sweep and candidate policy

ReaderBufferFileSizes generates 4 KiB, 64 KiB, 1 MiB and 16 MiB log fixtures outside timing and validates captures for each variant. It needs no shared fixture:

dotnet run -c Release --project test/perf/SelectString/SelectString.Engine.Benchmarks.csproj -- --filter '*ReaderBufferFileSizes*' --job short --affinity 1 --artifacts tmp/SelectString/buffer-sizes

The candidate uses a 16 KiB reader for seekable files of at least 1 MiB when the requested encoding is a runtime-provided UTF-8 encoding with replacement fallback and the scan is not a List/Quiet early-exit scan. Other cases keep 1 KiB. BOM detection remains enabled, so a BOM can still override the requested encoding. Strict decoding, custom encoding types and custom fallbacks retain their original read-ahead behavior. Explicit buffer arguments in the standalone benchmarks bypass selection, keeping the experiment independent of the candidate policy.

The 1 MiB gate is a conservative candidate supported by the size sweep, not a universal break-even point. See RESULTS-buffer-policy.md for end-to-end evidence.

End-to-end command

./test/perf/SelectString/Measure-SelectString.ps1 `
    -PwshPath ./tmp/SelectString/baseline/pwsh.exe,./tmp/SelectString/candidate/pwsh.exe `
    -Revision shipping,candidate `
    -Iterations 25 -WarmupIterations 15 -Affinity 1

Every build/scenario gets a separate process with a timeout. Startup, module loading, data preparation and warmups (three by default) are excluded. Throughput samples discard output with & $action > $null. A separate instrumented pass measures first-result latency and counts results. JSON includes all timing samples, process-wide managed allocation deltas, GC counts, peak working set, runtime, hardware and assembly hash. Peak working set includes setup and warmup; it is not incremental engine memory. PowerShell harness overhead is included in allocations and command timing, so use the standalone suite for small operation and allocation comparisons.

-Affinity is an optional processor affinity mask (zero leaves scheduling alone). The example pins workers to logical processor zero on Windows; use a mask allowed by your operating system, or omit it where process affinity is unsupported. Pin all compared builds to the same processor, particularly on hybrid CPUs. BenchmarkDotNet accepts the corresponding --affinity 1 option. Affinity and warmup count are recorded in the command results.

These are warm-cache tests. They cover strings, streamed records, 256 small files, a 100,000-line file, a 4 MiB line, sparse/dense/missing matches, multiple patterns, context, early/late termination, output modes and several encodings. Use -LineCount to scale the file workloads; fixture generation streams to disk and the record-input workload retains at most 10,000 records. The default run does not establish cold-storage performance or multi-gigabyte scalability. The JSON preserves failures/timeouts rather than reporting them as fast results. Counts alone do not establish correctness: use the differential comparison, and exclude incorrect scenarios from speedup claims. Formatting is excluded from the search throughput scenarios; Formatting measures context rendering separately. DenseContext and NoMatchContext isolate context costs at opposite match densities.

See RESULTS.md for the initial measurements and RESULTS-allocations.md for the allocation phase. The queue experiment and subsequent full workload validation use the null-redirection harness.

Focused Windows tracing

For the large-log scenarios, use the generation and capture commands in the section below. Existing DenseContext captures keep their previous defaults.

Run New-EtwCapture.ps1 from an elevated PowerShell after building shipping and the candidate. Shipping uses tmp/SelectString/baseline/pwsh.exe. Pass the build directory name under tmp/SelectString with -CandidateDirectory (tab completion is supported). When omitted, the newest eligible build is selected by the Utility assembly modification time, falling back to pwsh.exe modification time if the assembly is absent. Ties are resolved by directory name. Baseline, first-extraction, the old experiment and shipping directories are excluded from automatic selection/completion. Results and traces always use the labels shipping and candidate, independently of the selected directory; the selected executable path is printed before recording. The extraction build is no longer part of these captures. Run from the repository root:

./test/perf/SelectString/New-EtwCapture.ps1 -CandidateDirectory candidate

It writes separate DenseContext traces and benchmark JSON under tmp/SelectString/traces/<timestamp> and retains WPR stop logs. Do not run it alongside another recording. Reported event loss stops further captures.

SelectString.wprp captures system CPU samples with stacks and process/image events. CLR events are filtered to pwsh.exe, keyword mask 0x19 (GC, loader, JIT), at verbose level with allocation-tick event ID 10 excluded. This includes GC/pause activity, JIT starts and method-load metadata, but excludes heap dumps and inlining diagnostics. It does not attribute allocations by type. See the runtime event manifest for event levels and keywords. Each collector has 256 buffers of 256 KiB.

Recording begins before worker startup to capture method loads and includes warmup. The capture script enables -TraceMarkers and -SkipCounting so the ForEach-Object latency/count pass does not contaminate CPU captures. A plain for loop drives warmup. Ordinary measurement runs still include the separate counting pass unless explicitly skipped; skipped latency/count fields are null.

BenchmarkEvents.cs provides warmup, measured-iteration and counting boundaries. Marker calls are outside the timing/allocation samples; iteration regions also include the small measurement bookkeeping surrounding each timed call. The WPA profile loads SelectString.Regions.xml to expose these phases. Select measured iteration regions when comparing builds. Failed iterations may have an unmatched begin marker and must not be treated as successful samples.

JSON records Sink, WorkerPid, TraceMarkers and CountingMeasured. Null redirection discards streamed output without binding each object to a sink cmdlet. Null assignment was also tried, but retained output and produced substantial Gen 1/Gen 2 collection activity in dense scans. Historical reports used Out-Null or NullAssignment: rerun all compared builds with this harness rather than comparing new numbers directly against those old reports. Marker compilation and registration happen before warmup and only when requested; untraced runs do not load the provider. Actual ETW region pairing still needs validation in WPA. Filter analysis to the benchmark worker PID; other PowerShell processes also match the executable filter. Kernel sampling remains system-wide to expose competing CPU activity. Managed symbol coverage and event loss must be checked in the resulting trace. Traced timings are diagnostic, not benchmark claims.

Open traces with tools/performance/SelectString.wpaProfile for CPU stacks, engine-oriented stack tags and GC/JIT regions. See that directory's README for the opening command and interpretation limits.

Large synthetic application log

See RESULTS-log.md for initial measurements and validation.

Log scenarios automatically reuse tmp/SelectString/log-fixture. If neither the log nor its manifest exists, they generate the default 300 MiB fixture before workers or recording start. An incomplete or invalid fixture still fails validation rather than overwriting existing data. -LogFixture selects an explicit directory.

To generate a different size or location explicitly and reuse it across builds:

./test/perf/SelectString/New-LogFixture.ps1 -SizeMiB 300

./test/perf/SelectString/Measure-SelectString.ps1 `
    -PwshPath ./tmp/SelectString/baseline/pwsh.exe,./tmp/SelectString/candidate/pwsh.exe `
    -Revision shipping,candidate `
    -Scenario LogRare,LogSparse,LogMedium,LogBroad,LogDense,LogAllMatches,LogContext `
    -LogFixture ./tmp/SelectString/log-fixture `
    -WarmupIterations 3 -Iterations 5 -Affinity 1 -TimeoutSeconds 600 `
    -Artifacts ./tmp/SelectString/log-300-results

The default directory is tmp/SelectString/log-fixture; -Directory can select another location. Generation overwrites application.log and manifest.json in that directory. The file reaches at least the requested size, ending at a complete line. The manifest records actual bytes, line count, generator version and SHA256. The benchmark checks the hash before launching workers and never regenerates the shared log. Use -SizeMiB 1 and a separate directory for smoke checks.

Entries vary timestamp, worker, route, request ID, duration and message length (including Unicode). A four-digit bucket supplies multiple densities in the same file; queries capture named level, request and duration groups. Matching is case-sensitive. Density refers to matching lines, not capture count:

Scenario Matching lines Behavior
LogRare 0.01% Default detailed output
LogSparse 0.1% Default detailed output
LogMedium 1% Default detailed output
LogBroad 10% Default detailed output
LogDense 100% Default detailed output
LogAllMatches 100% Every seventeenth line has a second matching record
LogContext 0.1% Two lines of pre/post context

Percentages are approximate at a partial final bucket cycle. Bucket selection is regularly spaced; this is a reproducible synthetic workload, not evidence of production representativeness. It does not model clustered error bursts, multiline exceptions or cold storage. A representative anonymized log is still useful.

Before warmup, each worker streams a validation pass checking every result's line number, match count and named capture values against arithmetic expectations. Validation failures prevent a successful measurement. ValidatedResults and ValidatedMatches distinguish line counts from multiple regex matches. Existing differential tests remain the checks for complete context semantics. Validation also runs with -SkipCounting, outside measured ETW regions; it warms the cache and exercises the runtime before explicit warmup. Add-Type compilation is outside timing as well.

JSON includes the fixture manifest, MiB/s, allocation/GC samples and optional first-result latency. Peak working set includes setup, validation and counting; it is not incremental scan memory. Five calls/three explicit warmups are an initial exploration, not a stable performance claim; repeat comparisons and inspect runtime tiering before drawing conclusions.

For a focused capture, run from elevated PowerShell after generation:

./test/perf/SelectString/New-EtwCapture.ps1 `
    -Scenario LogContext `
    -WarmupIterations 3 -Iterations 5 -TimeoutSeconds 600

Choose another log scenario to vary density or exercise AllMatches. The script captures shipping and the selected candidate using the same file, with markers enabled and counting disabled. ETL filenames include the scenario. Select the measured regions (five for this example); validation and startup are also in the recording and must be excluded from timed-region analysis.

See RESULTS-traces.md for the large-log ETW findings and the host-dependent highlighting allocation difference.

Capturing multiple scenarios

New-EtwCapture.ps1 -Scenario accepts an array:

./test/perf/SelectString/New-EtwCapture.ps1 -Scenario LogSparse,LogAllMatches

For each scenario, shipping and candidate are recorded separately, in that order. The shared log is resolved/validated once before recording. Duplicate scenarios run once. A batch writes each scenario beneath its own directory in the capture folder, retaining separate ETLs, stop logs and benchmark JSON. Single-scenario runs retain their existing directory layout. Recording or measurement failure stops the batch after saving the active trace when possible.

Pattern families in actual cmdlet invocations

PatternFile scans a generated UTF-8 file; PatternRecords feeds the same lines through one pipeline invocation. Both use the corpus in PatternFamilies.cs, shared with the standalone benchmarks, and include per-invocation regex setup. Generation, Add-Type, validation and warmup are outside timing. Validation checks all returned line numbers, order, spans and capture groups against authored expectations. Output is discarded during throughput measurement.

./test/perf/SelectString/Measure-SelectString.ps1 `
    -PwshPath ./tmp/SelectString/baseline/pwsh.exe,./tmp/SelectString/candidate/pwsh.exe `
    -Revision shipping,current -Scenario PatternFile,PatternRecords `
    -PatternFamily DotStar -InputKind LongNearMiss -RecordCount 10000 `
    -CaseSensitive -Culture en-US -WarmupIterations 30 -Iterations 15 `
    -LatencyIterations 5 -Affinity 1 -TimeoutSeconds 300 `
    -Artifacts ./tmp/SelectString/pattern-dotstar-long

PatternFamily accepts Literal, DotStar, DotPlus, SqlDuration, OrderId, FileExtension, PrReference, Markdown and Assignment. InputKind accepts Match, Absent, NearMiss, LongNearMiss and Mixed. Mixed cycles equally through the four inputs, including 4 KiB near misses; this is a diagnostic distribution, not a claim about production traffic. RecordCount applies to both file and pipeline. Case-insensitive matching is the default; Culture defaults to en-US and is set before cmdlet construction in the isolated worker. The supplied ASCII fixtures are not themselves comprehensive culture tests; engine tests separately exercise Turkish I and preservation of construction-time culture.

JSON records the pattern, family, input kind, record count, case mode and actual culture. An absent-only workload validates zero results and has null first-result latency. Use a separate artifact directory for each parameter combination; reruns replace the files in that directory. Repeat with reversed executable order before interpreting small timing differences.

Background compilation probe

This is a standalone, benchmark-only prototype; it does not change the cmdlet. Build the benchmark project, then run the probe in separate processes:

dotnet build test/perf/SelectString/SelectString.Engine.Benchmarks.csproj -c Release
$probe = './test/perf/SelectString/bin/Release/net11.0/SelectString.Engine.Benchmarks.dll'
dotnet $probe --promotion-probe ./tmp/SelectString/promotion-one-core 1
dotnet $probe --promotion-probe ./tmp/SelectString/promotion-unrestricted 0

The final argument is a Windows affinity mask; zero retains OS affinity. Unrestricted affinity permits spare cores but does not pin specific physical cores. Use different directories for repeats: the probe replaces its per-case JSON files. Repeat in reversed affinity order before interpreting scheduling differences.

This is an exploratory measurement loop, not a BenchmarkDotNet job. It exercises one and 10,000 records for literal, wildcard and SQL patterns, with successful, absent-prefix and long-near-miss inputs. Each case rotates six modes through three warmup and nine measured rounds: interpreted, compiled, synchronous compilation after 128 records, immediate background constructor-only, immediate background with first execution, and background first execution after 128 records. The 128 record gate is a control for this experiment, not a proposed production threshold. The first-execution probe uses only the selected authored fixture input.

Each operation constructs its own engine. Background work captures construction culture and publishes a whole replacement between matching calls. A shared single worker slot rejects busy admissions without queuing. Completion suppresses work that has not begun and joins work already running. Worker failures are observed; the measurement fails if speculative compilation fails. Concurrency admission, faults, draining, culture, retained captures and the delayed gate have unit tests.

JSON retains foreground and total wall time, process CPU, managed allocations, first-result latency, admission/rejection, whether compilation started, promotion record index and time observed after that matching call, and unused compilations. Totals include draining the worker; no admitted work carries into the next call. Process CPU is coarse, especially for sub-millisecond calls, and managed allocation excludes native generated code. Foreground time deliberately excludes drain time; it does not claim a cmdlet could abandon workers without lifecycle consequences. Match counts are checked every call, with corpus span/capture validation before measurement. These results omit pipeline/file/output costs and concurrent callers.

Isolated cmdlet promotion experiment

The files under experiment/ are applied only to a separate worktree. The main production cmdlet and engine remain unchanged. Starting from commit 2f9c17760, create the worktree and apply the narrow matching/lifecycle hook:

$sourceRoot = 'E:/repos/PowerShell.worktrees/SelectString-promotion-experiment'
git worktree add --detach $sourceRoot 2f9c17760
$patch = (Resolve-Path ./test/perf/SelectString/experiment/CmdletPromotion.patch).Path
git -C $sourceRoot apply $patch
$utility = Join-Path $sourceRoot 'src/Microsoft.PowerShell.Commands.Utility/commands/utility'
Copy-Item ./test/perf/SelectString/experiment/SelectStringPromotion.cs $utility
$background = Get-Content ./test/perf/SelectString/BackgroundCompilation.cs -Raw
$background = $background.Replace('_active.TryMatch(', '_active.TryMatchCore(')
$background = $background.Replace('replacement.TryMatch(', 'replacement.TryMatchCore(')
[IO.File]::WriteAllText((Join-Path $utility 'BackgroundCompilation.cs'), $background)
./test/SelectString/Build-Comparison.ps1 -SourceRoot $sourceRoot `
    -OutputDirectory (Join-Path $PWD 'tmp/SelectString/regex-background')

The adapter calls the unhooked core to avoid recursively re-entering promotion. The existing reader, trackers and output code stay in the scanner. Workers capture argument values rather than accessing cmdlet state. The build helper generates resources/types. Preserve existing baseline/current/compiled binaries.

The experimental build defaults to background constructor-only compilation. Measure-SelectString.ps1 -PromotionMode selects interpreted, compiled, background, background-warm, delayed-synchronous or delayed-background-warm in an isolated worker. The delayed modes use the exploratory 128-record control. Warm modes take the scenario's first authored input; they are for these fixtures only. Internally the experiment reads SELECTSTRING_PROMOTION_MODE and SELECTSTRING_PROMOTION_WARM_INPUT. These are not production settings.

./tmp/SelectString/regex-background/pwsh.exe -NoProfile `
    -File ./test/SelectString/Test-PromotionLifecycle.ps1
./test/perf/SelectString/Measure-SelectString.ps1 `
    -PwshPath ./tmp/SelectString/regex-background/pwsh.exe -Revision background `
    -PromotionMode background -Scenario PatternFile,PatternRecords,PipelineStop,FileStop `
    -PatternFamily DotStar -InputKind LongNearMiss -CaseSensitive -RecordCount 10000 `
    -WarmupIterations 30 -Iterations 15 -LatencyIterations 5 -Affinity 1 `
    -TimeoutSeconds 300 -Artifacts ./tmp/SelectString/promotion-cmdlet-example

The lifecycle test holds an admitted compiler, checks rejected concurrent callers, then exercises downstream termination, injected worker failure and cancellation. It verifies exactly-once disposal and no remaining worker. Its barriers/fault flag are null/false during measurement. The experimental diagnostics type is not a supported API. Disposal drains work; stop suppresses further admission/publication. This deliberately measures joined cleanup, not a final production cancellation policy. Regex compilation already in progress cannot be forcibly cancelled.

Worker JSON includes cumulative deltas for completed invocations, admitted/started/ rejected/promoted/faulted work and drain milliseconds. Timed calls include disposal. Warmup and separate latency passes are excluded from these counters. Unexpected worker faults or workers remaining after timing fail the measurement. CPU is measured over the full timing loop, including harness bookkeeping, with CpuWindowMs, CpuMs and CpuMsPerInvocation. Short windows remain coarse; do not treat small CPU differences as precise. Modes are null for non-experimental builds.

The paired 300 MiB sparse/dense capture comparison is recorded in RESULTS-background-log.md, including CPU, first-result latency and promotion/cleanup validation.

One file versus many small files

PatternSmallFiles splits the pattern corpus's RecordCount records into files of LinesPerFile lines (default 16), with a possibly shorter final file. One Select-String invocation receives the complete ordered path array. Fixture and path-array construction happen before timing. Validation checks each result's file path, local line number, input text and captures, including file boundaries. JSON records PatternFileCount and PatternLinesPerFile.

The same scenario can represent several large files by increasing LinesPerFile. It supports -PatternOutput Context, with validation bounded by each file's edges. For a constrained consumer, add -SinkDelayMilliseconds 1 -SinkDelayEvery 128. Throughput samples then include a ForEach-Object consumer that discards results and sleeps after every 128 results. This measures downstream binding and blocking as well as scanning; actual sleep duration depends on the scheduler. The JSON identifies this sink and its settings. Validation and first-result measurements use the ordinary undelayed consumer, so first-result latency does not describe the delayed sink. Leave affinity unrestricted when testing multi-core scaling. The first ordered-parallel comparison is recorded in RESULTS-parallel-files.md. The bounded batch-size follow-up is in RESULTS-parallel-batches.md; it retains the smaller default because larger batches did not establish a repeatable wall-time win.

./test/perf/SelectString/Measure-SelectString.ps1 `
    -PwshPath ./tmp/SelectString/regex-background/pwsh.exe -Revision background `
    -PromotionMode background -Scenario PatternFile,PatternSmallFiles `
    -PatternFamily SqlDuration -InputKind Mixed -CaseSensitive `
    -RecordCount 4096 -LinesPerFile 16 -LineCount 16 `
    -WarmupIterations 30 -Iterations 15 -LatencyIterations 5 -Affinity 5 `
    -TimeoutSeconds 300 -Artifacts ./tmp/SelectString/small-files-example

The two scenarios use identical records, partitioned as one file or 256 files. Warm-cache measurements include opening/closing files and cmdlet path handling; they do not model cold storage. Choose an affinity appropriate to your machine.

Results for literal hits and mixed SQL captures/near misses are in RESULTS-background-small-files.md.

Pattern output modes

PatternFile accepts -PatternOutput with Matches, Context, Raw, List or Quiet (one value per run; default Matches). Context requests two lines before/after each match. The other pattern scenarios currently require Matches. Validation runs outside timing and checks logical context contents, raw strings, first-match captures for List, or the true/false Quiet result. A false Quiet result is still one output object, so its first-result latency measures completion of the scan.

./test/perf/SelectString/Measure-SelectString.ps1 `
    -PwshPath ./tmp/SelectString/regex-background/pwsh.exe -Revision background `
    -PromotionMode background -Scenario PatternFile -PatternOutput Quiet `
    -PatternFamily SqlDuration -InputKind LongNearMiss -CaseSensitive `
    -RecordCount 4096 -LineCount 16 -WarmupIterations 20 -Iterations 15 `
    -LatencyIterations 5 -Affinity 5 -TimeoutSeconds 300 `
    -Artifacts ./tmp/SelectString/quiet-full-scan-example

Use Mixed for a match on the first line (early List/Quiet termination) and LongNearMiss for a complete scan without matches. The same Mixed corpus with Context/Raw exercises sustained matching and output. Neither input is a proxy for every pattern's compilation cost.

See RESULTS-background-output-modes.md for the paired output-mode measurements and unused-compilation accounting.

Multiple patterns

-PatternSet Single (default) uses the selected corpus pattern alone. TargetFirst and TargetLast add the corpus OrderId, PrReference and Assignment patterns, placing the target before or after them. These controls support Literal and SqlDuration inputs only, where the added patterns do not match. Validation checks the winning MatchInfo.Pattern as well as spans/captures and output order. Worker JSON records the complete ordered SearchPatterns array and PatternSet.

./test/perf/SelectString/Measure-SelectString.ps1 `
    -PwshPath ./tmp/SelectString/regex-background/pwsh.exe -Revision background `
    -PromotionMode background -Scenario PatternFile -PatternFamily SqlDuration `
    -PatternSet TargetLast -InputKind Mixed -CaseSensitive -RecordCount 4096 `
    -LineCount 16 -WarmupIterations 20 -Iterations 15 -LatencyIterations 5 `
    -Affinity 5 -TimeoutSeconds 300 -Artifacts ./tmp/SelectString/multi-example

All patterns are constructed once per invocation. The explicit background warm probe still calls matching once on the first authored input; first-pattern success can prevent later patterns from being exercised. It is not an all-pattern JIT warmup. These controlled sets do not cover overlapping successful patterns.

See RESULTS-background-multiple-patterns.md for pattern-order comparisons, startup costs and unused-work counters.

Matching-work admission probe

The test-only MatchingWorkGate admits background preparation at the next input after sampled matching wall time reaches a threshold. Time between matching calls (including reading and downstream output) is excluded. Preemption and GC within a sampled call still count. Sampling does not extrapolate unobserved work and can miss expensive inputs. Admission/rejection is attempted once, and timing stops after that decision or a stop request. Final-call threshold crossing alone starts no worker. Joined disposal still accounts for admitted work.

dotnet build ./test/perf/SelectString/SelectString.Engine.Benchmarks.csproj -c Release
dotnet ./test/perf/SelectString/bin/Release/net11.0/SelectString.Engine.Benchmarks.dll `
    --promotion-probe ./tmp/SelectString/admission-example 5 admission

The optional admission profile compares direct interpretation, a wrapper without admission, timing-only controls, the previous compilation controls, and work gates timing every call or every 16th call. It uses an exploratory 0.5 ms observed-time threshold, not a selected production policy. Output includes admission/promotion record indices, observed matching time and sample counts. Warm gate variants still use explicitly authored fixture input. The profile covers Literal/SqlDuration, matching/long-near-miss input and 1/4096 records. Choose affinity for your machine. The default profile without admission retains the previous six-mode matrix.

Initial overhead/admission findings are in RESULTS-admission.md. The sampled gate still needs adversarial input testing before cmdlet integration.