PR Summary

This PR performs special handling for a particular json use-case where the response body is an array. This detects that situation and deserializes the array one object at a time. This greatly improves runtime characteristics and reduces memory usage when the json is an array of objects while preventing us from having to implement our own full json parser.

This PR does not improve performance of parsing a single large dictionary.

PR Context

ConvertFrom-Json is a core cmdlet for anyone using Powershell for Web APIs. The Cmdlet has very poor runtime characteristics. It takes a long time to deserialize json compared to other deserializing in other languages and it uses a lot of memory to deserialize (15x the size of the json).

This PR on its own increases memory usage as compared to master. Combined with #15684 this PR cuts memory usage and runtime by more than half.

Benchmark for 100MB JSON file:

Master

• Memory Usage: 1079.3671875 MB
• Runtime: 7256.4268 ms

ThisPR Alone

• Memory Usage: 1477.83203125 MB
• Runtime: 6574.552 ms

#15684

• Memory Usage: 1077.58203125 MB
• Runtime: 3885.0489 ms

ThisPR + #15684

• Memory Usage: 451.390625 MB
• Runtime: 3077.3673 ms

PR Checklist

• PR has a meaningful title
  • Use the present tense and imperative mood when describing your changes
• Summarized changes
• Make sure all .h, .cpp, .cs, .ps1 and .psm1 files have the correct copyright header
• This PR is ready to merge and is not Work in Progress.
  • If the PR is work in progress, please add the prefix WIP: or [ WIP ] to the beginning of the title (the WIP bot will keep its status check at Pending while the prefix is present) and remove the prefix when the PR is ready.
• Breaking changes
  • None
  • OR
  • Experimental feature(s) needed
    • Experimental feature name(s):
• User-facing changes
  • Not Applicable
  • OR
  • Documentation needed
    • Issue filed:
• Testing - New and feature
  • N/A or can only be tested interactively
  • OR
  • Make sure you've added a new test if existing tests do not effectively test the code changed
• Tooling
  • I have considered the user experience from a tooling perspective and don't believe tooling will be impacted.
  • OR
  • I have considered the user experience from a tooling perspective and opened an issue in the relevant tool repository. This may include:
    • Impact on PowerShell Editor Services which is used in the PowerShell extension for VSCode
      (which runs in a different PS Host).
      • Issue filed:
    • Impact on Completions (both in the console and in editors) - one of PowerShell's most powerful features.
      • Issue filed:
    • Impact on PSScriptAnalyzer (which provides linting & formatting in the editor extensions).
      • Issue filed:
    • Impact on EditorSyntax (which provides syntax highlighting with in VSCode, GitHub, and many other editors).
      • Issue filed: