ConvertFrom-JSON high memory consumption #7698
Description
Steps to reproduce
$JSON = @"
{
"configGlossary:installationAt": "Philadelphia, PA",
"configGlossary:adminEmail": "ksm@pobox.com",
"configGlossary:poweredBy": "Cofax",
"configGlossary:poweredByIcon": "\/images\/cofax.gif",
"configGlossary:staticPath": "\/content\/static",
"templateProcessorClass": "org.cofax.WysiwygTemplate",
"templateLoaderClass": "org.cofax.FilesTemplateLoader",
"templatePath": "templates",
"templateOverridePath": "",
"defaultListTemplate": "listTemplate.htm",
"defaultFileTemplate": "articleTemplate.htm",
"useJSP": false,
"jspListTemplate": "listTemplate.jsp",
"jspFileTemplate": "articleTemplate.jsp",
"cachePackageTagsTrack": 200,
"cachePackageTagsStore": 200,
"cachePackageTagsRefresh": 60,
"cacheTemplatesTrack": 100,
"cacheTemplatesStore": 50,
"cacheTemplatesRefresh": 15,
"cachePagesTrack": 200,
"cachePagesStore": 100,
"cachePagesRefresh": 10,
"cachePagesDirtyRead": 10,
"searchEngineListTemplate": "forSearchEnginesList.htm",
"searchEngineFileTemplate": "forSearchEngines.htm",
"searchEngineRobotsDb": "WEB-INF\/robots.db",
"useDataStore": true,
"dataStoreClass": "org.cofax.SqlDataStore",
"redirectionClass": "org.cofax.SqlRedirection",
"dataStoreName": "cofax",
"dataStoreDriver": "com.microsoft.jdbc.sqlserver.SQLServerDriver",
"dataStoreUrl": "jdbc:microsoft:sqlserver:\/\/LOCALHOST:1433;DatabaseName=goon",
"dataStoreUser": "sa",
"dataStorePassword": "dataStoreTestQuery",
"dataStoreTestQuery": "SET NOCOUNT ON;select test='test';",
"dataStoreLogFile": "\/usr\/local\/tomcat\/logs\/datastore.log",
"dataStoreInitConns": 10,
"dataStoreMaxConns": 100,
"dataStoreConnUsageLimit": 100,
"dataStoreLogLevel": "debug",
"maxUrlLength": 500
}
"@
$Count = 39999
$jsonarr = "[" + (($json + ",") * $Count) + $json + "]"
[system.gc]::Collect()
write-host ""
write-host ""
write-host "JSON -- Objects: $($Count + 1)"
write-host "JSON -- Length: $($jsonarr.length / 1024 / 1024 ) Megabytes"
$BeforeMemory = $((get-process -pid $PID).WorkingSet)
[void] (convertfrom-json $jsonarr)
$AfterMemory = $((get-process -pid $PID).WorkingSet)
$MemoryDiff = $AfterMemory - $BeforeMemory
write-host "Memory Usage -- Before: $($BeforeMemory / 1024 / 1024) Megabytes"
write-host "Memory Usage -- After: $($AfterMemory / 1024 / 1024) Megabytes"
write-host "Memory Usage -- Difference: $($MemoryDiff / 1024 / 1024) Megabytes"
write-host "Memory Usage -- Multiple: $($MemoryDiff / $jsonarr.length)"
output:
JSON -- Objects: 40000
JSON -- Length: 67.4819955825806 Megabytes
Memory Usage -- Before: 210.8359375 Megabytes
Memory Usage -- After: 1240.90625 Megabytes
Memory Usage -- Difference: 1030.0703125 Megabytes
Memory Usage -- Multiple: 15.2643724241892
Actual behavior
Powershell Core ends up with 1 GB of memory used and peaks to 1.6 GB of memory utilization during convertfrom-json
Expected behavior
Powershell Core shouldn't need 1.6 gigabytes of memory to deserialize a 65MB JSON file.
Environment data
Name Value
---- -----
PSVersion 6.1.0-preview.3
PSEdition Core
GitCommitId v6.1.0-preview.3
OS Microsoft Windows 10.0.17134
Platform Win32NT
PSCompatibleVersions {1.0, 2.0, 3.0, 4.0...}
PSRemotingProtocolVersion 2.3
SerializationVersion 1.1.0.1
WSManStackVersion 3.0