Skip to content

Get-Content -ReadCount 0 combined with -Last / -Tail seems to read ALL lines internally #11830

@mklement0

Description

@mklement0

Get-Content -ReadCount 0 is a convenient way to request that all lines be read at once, into an array and to have that array be output as a single object to the success stream.

When combined with -First aka -TotalCount aka -Head, this sensibly allocates an array for and outputs only the requested number of lines from the beginning, not for all lines.
Note: Meaningfully combining -First with -ReadCount 0 was only recently implemented, in #10749.

While combining -Last aka -Tail with -ReadCount 0 is functional, performance timings suggest that all lines are needlessly being read into an array behind the scenes, before only the sub-array of interest is output.

In other words: while -Tail <n> -ReadCount 0 should be the same as -Tail <n> -ReadCount <n> (explicitly setting the the read count to the same number as the number of trailing lines requested), it currently isn't in terms of performance and memory use.

Steps to reproduce

# Create a temporary file with 1 million lines
$f = [IO.Path]::GetTempFileName(); (, 'foo') * 1e6 > $f
# Warm up the cache.
$tmp = gc $f -ReadCount 0

# Read 1000 lines from the end, as a single array
$n = 1000
{ $v = gc $f -Tail $n -ReadCount $n },
{ $v = gc $f -Tail $n -ReadCount 0  }, 
{ $v = gc $f          -ReadCount 0  } <# control: read all lines #> |  % {
  "$_`: " + (Measure-Command $_).TotalSeconds 
}

Remove-Item $f

Expected behavior

-Tail $n -ReadCount $n and -Tail $n -ReadCount 0 should perform virtually the same and should be faster than -ReadCount 0 by itself.

Actual behavior

-Tail $n -ReadCount 0 is not only slower than -Tail $n -ReadCount $n , but also slower than -ReadCount 0 by itself, suggesting that all lines were read behind the scenes.

 $v = gc $f -Tail $n -ReadCount $n : 0.1515997
 $v = gc $f -Tail $n -ReadCount 0  : 0.2118531
 $v = gc $f          -ReadCount 0  : 0.209674

Environment data

PowerShell Core 7.0.0-rc.2

Metadata

Metadata

Assignees

No one assigned

    Labels

    Issue-Questionideally support can be provided via other mechanisms, but sometimes folks do open an issue to get aResolution-FixedThe issue is fixed.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions