@mklement0 can you look at the test failures?

@SteveL-MSFT, I'll focus on fixing the fundamental problem first. A preliminary investigation showed that the problem was just one of duplicates in the output, strangely; applying Select-Object -Unique before counting / comparing made the problem go away.

@SteveL-MSFT, looking into this further my conclusion is that this PR has simply unmasked an unrelated problem that should be investigated separately.

For now, I've restored the semantics of the original test by passing -PowerShellVersion 5.0 to the Get-FormatData commands in the original tests, which makes them succeed.

As for what may be going on:

The current format-data definitions contain duplicates:

Some types are covered both by a dedicated definition (where .TypeNames references a single type) and by a multi-type definition (where .TypeNames contains multiple type names).

$dataWithMultipleTypes = Get-FormatData -PowerShellVersion 5.1 | Where { $_.TypeNames.Count -gt 1 }
$dataWithSingleType = Get-FormatData -PowerShellVersion 5.1 | Where { $_.TypeNames.Count -eq 1 }

# Find all single-type definitions also covered by multiple-type definitions.
$dataWithSingleType | ? { foreach ($fd in $dataWithMultipleTypes) { if ($fd.TypeNames -contains $_) { return $true } } }

This currently yields:

TypeNames                                                        FormatViewDefinition
---------                                                        --------------------
{System.Security.Cryptography.X509Certificates.X509Certificate2} {ThumbprintTable}
{System.Management.Automation.ApplicationInfo}                   {ApplicationInfo, System.Management.Automation.ApplicationInfo}
{System.Management.Automation.ScriptInfo}                        {ScriptInfo, System.Management.Automation.ScriptInfo}
{System.Management.Automation.ExternalScriptInfo}                {ExternalScriptInfo}
{System.Management.Automation.FunctionInfo}                      {FunctionInfo}
{System.Management.Automation.FilterInfo}                        {FilterInfo}
{System.Management.Automation.AliasInfo}                         {AliasInfo, System.Management.Automation.AliasInfo}
{System.Management.Automation.CmdletInfo}                        {System.Management.Automation.CmdletInfo}
{System.Management.Automation.PSDriveInfo}                       {System.Management.Automation.PSDriveInfo}

• Is there a good reason for this duplication?
• If so, what are the precedence rules, and is Update-FormatData expected to load the duplicates too?

@mklement0 adding | % { Get-FormatData -PowerShellVersion 5.1 -TypeName $_.TypeNames } to your example, it seems that the duplication is for a base type and derived types. So that seems to be fine. As for ordering, with the exception of Out-of-Band types (like errors), it should go in order of the psobject.typenames list.

@SteveL-MSFT

There is genuine duplication, with dormant duplicates among the built-in formats - see #11307

I suspect that this duplication ultimately causes the round-trip Get-FormatData test to fail with -PowerShellVersion 5.1 (or this fix in place), but that's a separate issue - see #11308

@SteveL-MSFT, are there concerns around the public method added to InternalTestHooks? Apart from that, I think this is ready to be merged.

@mklement0 Please have a look at the merge conflicts

@adityapatwardhan, I've resolved the merge conflict.

The PowerShell-CI-static-analysis failure seems like a transient one ("SocketException: Network is unreachable") - not sure how to restart the test.

@PoshChan please retry static

@SteveL-MSFT, successfully started retry of PowerShell-CI-static-analysis

@mklement0 There seems to be another merge conflict. Please have a look.

@adityapatwardhan , I've resolved the (trivial) conflict in favor of preferring a whitespace after the unary form of the , operator (, <operand> rather than ,<operand>), which looks cleaner to me (it's also what auto-formatting in VSCode does).

@mklement0 Thanks for your contribution!

🎉v7.1.0-preview.2 has been released which incorporates this pull request.:tada:

Handy links:

• Release Notes

@SteveL-MSFT

Please see the update I posted yesterday that still exists in PS 7.2.0 to an issue I previously reported about issues with Get-FormatData / Export-FormatData / Update-FormatData.

Here's the link with a detailed example that is 100% reproducible with PS 7.2.0

#9300

A quick summary would be that:

A roundtrip of format data using Get-FormatData / Export-FormatData / Update-FormatData results in broken output.

The 2 main issues are:

When format data has a custom control it is not returned by Get-FormatData and therefore is lost on the roundtrip.

When format data contains more than 1 TypeName, Get-FormatData seems to get both but ExportFormat data only exports the first 1 so additional types are lost in the roundtrip.

This can be seen in the PS source code

PowerShell-7.2.0/src/System.Management.Automation/FormatAndOutput/DefaultFormatters/FileSystem_format_ps1xml.cs

where the custom control is created and shared between views and where a view has more than 1 TypeName.

It is frustrating to see these problems still exist in the latest PS when they have existed and reported so long ago.

Get-FormatData / Export-FormatData / Update-FormatData are GREAT feaures but the *.format.ps1xml files should never have been removed until the replace actually was fully functional.

Fixing Export-FormatData to include all TypeNames seems like it should be a fairly easy fix since Get-FormatData appears to have them.

Fixing the CustomControl situation seems more complicated since it is shared.

Here's a thought:

DirectoryInfo / FileInfo objects have the PSParentPath note property which looks like this:

string PSParentPath=Microsoft.PowerShell.Core\FileSystem::/home/joe

Add a new note property ParentPath

string ParentPath=/home/joe

Then all of the places that are using the shared custom control for their GroupBy could just be changed to reference ParentPath and there would no longer be a need for the custom control!

        <GroupBy>
            <PropertyName>ParentPath</PropertyName>
        </GroupBy>

This seems like a fairly easy change and by eliminating the shared custom control you remove the more difficult situation to fix.

What do you think?

Joe