Skip to content

Import-CSV has very high filesize-to-memory consumption ratio of around 50 #7603

@bergmeister

Description

@bergmeister

Steps to reproduce

Importing a 150 MB CSV file with 50k rows, 700 columns and 5 +/- 3 characters per entry results in a memory consumption of 7GB of the PowerShell process. This is a ratio of around 50 when compared to the file size. I have seen this ratio also with a similar file of smaller size (1.5MB) and therefore think this is a more generic issue not specific to the CSV file structure itself.

$csv = Import-Csv $pathToCsvFiile

Note that this does not happen if I do not assign to a variable or pipe to Out-Null

Expected behavior

Filesize to memory ratio is reasonable (around 10)

Actual behavior

Filesize to memory ratio is around 50

Environment data

Latest daily build and RC1

> $PSVersionTable
Name                           Value
----                           -----
PSVersion                      6.1.0-preview.761
PSEdition                      Core
GitCommitId                    6.1.0-preview.761
OS                             Microsoft Windows 6.3.9600
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0


Name                           Value
----                           -----
PSVersion                      6.1.0-rc.1
PSEdition                      Core
GitCommitId                    6.1.0-rc.1
OS                             Microsoft Windows 6.3.9600
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

Metadata

Metadata

Assignees

No one assigned

    Labels

    Issue-Enhancementthe issue is more of a feature request than a bug

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions