Use Source Code Generator for PSVersionInfo class #15603

iSazonov · 2021-06-18T13:12:20Z

PR Summary

The custom source generator allows us to exclude reflection and to directly get current PowerShell version from MSBuild.

PerfView doesn't work well on current .Net Preview. A fix has just been merged in .Net Runtime repo for this. I hope we get it in .Net 6.0 Preview6 or Preview7. So today I can not share reliable result from PerfView to confirm perf win. Obviously using static string is more faster then reflection.

PR Context

Contribute to #13540
Related #14268

PR Checklist

iSazonov · 2021-06-18T15:57:34Z

It seems test fails are not related to the PR - pwsh.exe has Preview.8 in product version instead of Preview.7.

@rjmholt @TravisEz13 Could you please look the issue?

rjmholt · 2021-06-18T17:37:56Z

@iSazonov I opened #15607

src/SourceGenerators/PSVersionInfoGenerator/PSVersionInfoGenerator.cs

src/System.Management.Automation/engine/PSVersionInfo.cs

src/SourceGenerators/PSVersionInfoGenerator/PSVersionInfoGenerator.cs

iSazonov · 2021-06-25T09:45:09Z

@rjmholt I resolved the issue with release tag. It turned out that ci.psm1 always assigns ReleaseTag property directly to dotnet.

ghost · 2021-07-02T14:00:39Z

This pull request has been automatically marked as Review Needed because it has been there has not been any activity for 7 days.
Maintainer, please provide feedback and/or mark it as Waiting on Author

src/SourceGenerators/PSVersionInfoGenerator/PSVersionInfoGenerator.cs

TravisEz13

Pending @wg-security review

…tor it

daxian-dbw

I integrated the FastParsePSVersion change from #18210 to the source generator, and also refactored the code. After the change, the static constructor of PSVersionTable only takes 2-3 ms on my dev machine (release build with R2R images, using the release tag v7.3.0 to mimic a stable release). It was 24-30 ms without the changes of this PR:

## Measured by wrapping the static constructor with `StopWatch`.

C:\>E:\arena\source\PowerShell\src\powershell-win-core\bin\Release\net7.0\win7-x64\publish\pwsh.exe -noprofile
2.5478
PowerShell 7.3.0
PS C:\> exit

C:\>E:\arena\source\PowerShell\src\powershell-win-core\bin\Release\net7.0\win7-x64\publish\pwsh.exe -noprofile
2.5993
PowerShell 7.3.0
PS C:\> exit

C:\>E:\arena\source\PowerShell\src\powershell-win-core\bin\Release\net7.0\win7-x64\publish\pwsh.exe -noprofile
2.5842
PowerShell 7.3.0
PS C:\> exit

src/System.Management.Automation/System.Management.Automation.csproj

- Turn off EmitCompilerGeneratedFiles - Remove first blank line in generated code

...stem.Management.Automation/SourceGenerators/PSVersionInfoGenerator/PSVersionInfoGenerator.cs

…ot affect re-building

daxian-dbw · 2022-10-06T19:20:37Z

src/System.Management.Automation/System.Management.Automation.csproj

+    <!-- we persist source generator files under 'gen' folder so that they are visible to IDEs -->
+    <EmitCompilerGeneratedFiles>true</EmitCompilerGeneratedFiles>
+    <CompilerGeneratedFilesOutputPath>gen\SourceGenerated</CompilerGeneratedFilesOutputPath>
+  </PropertyGroup>


Generate the file, so VSCode and Visual Studio can always resolve the generated members.
Those files will be removed when building again, by RemoveDir in PowerShell.Common.props.

I pushed a commit a day before (but you reverted now) and removed this old workaround because all works well in VS Code. We have no need to emit generated files.
Perhaps you need re-configure your local environment.
I use VS Code 1.72 and C# extension 1.25

You didn't see my reply to your comment? It's here: #15603 (comment), and quoted below:

It doesn't work in the following scenarios, for both VSCode and Visual Studio, which are quite common in development:

Run Start-PSBuild first, then open System.Management.Automation in VSCode

Open System.Management.Automation in VSCode to use an existing build, like from yesterday.

Sorry, too many comments already....

daxian-dbw · 2022-10-06T19:22:43Z

PowerShell.Common.props

    </PropertyGroup>

+    <!-- Remove existing generated files by source generators because new source code will soon be generated when the build starts -->
+    <RemoveDir Directories="gen\SourceGenerated" Condition="Exists('gen\SourceGenerated')" />


The path gen\SourceGenerated is duplicated in here and System.Management.Automation.csproj. I tried creating a property holding the path in the PropertyGroup right above, and use it in RemoveDir and CompilerGeneratedFilesOutputPath. However, it works in RemoveDir, but not in CompilerGeneratedFilesOutputPath, and I cannot figure out why. I'd love if someone can make it work.

The review was done a long time ago. Will request to re-review.

daxian-dbw · 2022-10-06T21:11:33Z

@TravisEz13 and @PaulHigin, can you please review the changes again and re-evaluate the "code injection" concern?

Here is the doc (and the sample code) about the source generator: https://learn.microsoft.com/en-us/dotnet/csharp/roslyn-sdk/source-generators-overview

The arguments used to format the source template in PSVersionInfoGenerator.cs are from the build properties defined in PowerShell.Common.props.

PowerShell.Common.props

...stem.Management.Automation/SourceGenerators/PSVersionInfoGenerator/PSVersionInfoGenerator.cs

....Management.Automation/SourceGenerators/PSVersionInfoGenerator/PSVersionInfoGenerator.csproj

src/System.Management.Automation/engine/PSVersionInfo.cs

iSazonov

LGTM.

PaulHigin · 2022-10-10T21:06:17Z

@WG-Security
The security group is Ok with this, because for release builds the generated version comes from a trusted source (releaseTag).

daxian-dbw

LGTM

pull-request-quantifier-deprecated · 2022-10-11T18:17:05Z

This PR has 115 quantified lines of changes. In general, a change size of upto 200 lines is ideal for the best PR experience!

Quantification details

Label      : Medium
Size       : +98 -17
Percentile : 43%

Total files changed: 5

Change summary by file extension:
.props : +2 -0
.cs : +62 -16
.csproj : +34 -1

Change counts above are quantified counts, based on the PullRequestQuantifier customizations.

Why proper sizing of changes matters

Optimal pull request sizes drive a better predictable PR flow as they strike a
balance between between PR complexity and PR review overhead. PRs within the
optimal size (typical small, or medium sized PRs) mean:

Fast and predictable releases to production:
- Optimal size changes are more likely to be reviewed faster with fewer
  iterations.
- Similarity in low PR complexity drives similar review times.
Review quality is likely higher as complexity is lower:
- Bugs are more likely to be detected.
- Code inconsistencies are more likely to be detected.
Knowledge sharing is improved within the participants:
- Small portions can be assimilated better.
Better engineering practices are exercised:
- Solving big problems by dividing them in well contained, smaller problems.
- Exercising separation of concerns within the code changes.

What can I do to optimize my changes

Use the PullRequestQuantifier to quantify your PR accurately
- Create a context profile for your repo using the context generator
- Exclude files that are not necessary to be reviewed or do not increase the review complexity. Example: Autogenerated code, docs, project IDE setting files, binaries, etc. Check out the Excluded section from your prquantifier.yaml context profile.
- Understand your typical change complexity, drive towards the desired complexity by adjusting the label mapping in your prquantifier.yaml context profile.
- Only use the labels that matter to you, see context specification to customize your prquantifier.yaml context profile.
Change your engineering behaviors
- For PRs that fall outside of the desired spectrum, review the details and check if:
  - Your PR could be split in smaller, self-contained PRs instead
  - Your PR only solves one particular issue. (For example, don't refactor and code new features in the same PR).

How to interpret the change counts in git diff output

One line was added: +1 -0
One line was deleted: +0 -1
One line was modified: +1 -1 (git diff doesn't know about modified, it will
interpret that line like one addition plus one deletion)
Change percentiles: Change characteristics (addition, deletion, modification)
of this PR in relation to all other PRs within the repository.

Was this comment helpful? 👍 :ok_hand: :thumbsdown: (Email)
Customize PullRequestQuantifier for this repository.

daxian-dbw · 2022-10-11T18:27:08Z

@iSazonov Just FYI, I re-did the conflict resolution because you accidentally brough back the PSCompatibleVersions property, which was deleted in the other PR.

iSazonov · 2022-10-12T04:57:26Z

@daxian-dbw Thanks for working on the SG!

Since you mentioned saving build time I think you could be interesting in SG for TypeGen.

iSazonov · 2022-10-12T05:18:29Z

@iSazonov Just FYI, I re-did the conflict resolution because you accidentally brough back the PSCompatibleVersions property, which was deleted in the other PR.

Gihub did show only one merge conflict for

       internal static string ProductVersion { get; }
        internal static string GitCommitId
        {
            get
            {
                return (string)s_psVersionTable[PSGitCommitIdName];
            }
        }

but did not marked PSCompatibleVersions. I wonder why. 😕

daxian-dbw · 2022-10-12T16:25:07Z

Since you mentioned saving build time I think you could be interesting in SG for TypeGen.

I don't know if that would save build time, I doubt it :)
Again, it's not about saving build time, but more about not doing things that are not needed.

iSazonov added the CL-Performance Indicates that a PR should be marked as a performance improvement in the Change Log label Jun 18, 2021

iSazonov requested review from SteveL-MSFT, daxian-dbw and rjmholt June 18, 2021 13:12

iSazonov requested a review from adityapatwardhan as a code owner June 18, 2021 13:12

iSazonov assigned rjmholt Jun 18, 2021

iSazonov requested review from TravisEz13 and anmenaga as code owners June 18, 2021 13:12

rkeithhill reviewed Jun 18, 2021

View reviewed changes

src/SourceGenerators/PSVersionInfoGenerator/PSVersionInfoGenerator.cs Outdated Show resolved Hide resolved

rkeithhill reviewed Jun 18, 2021

View reviewed changes

src/System.Management.Automation/engine/PSVersionInfo.cs Outdated Show resolved Hide resolved

rkeithhill reviewed Jun 18, 2021

View reviewed changes

src/System.Management.Automation/engine/PSVersionInfo.cs Outdated Show resolved Hide resolved

rjmholt reviewed Jun 18, 2021

View reviewed changes

src/SourceGenerators/PSVersionInfoGenerator/PSVersionInfoGenerator.cs Outdated Show resolved Hide resolved

iSazonov force-pushed the sg-psversion branch 4 times, most recently from cbb19d0 to 5abcd36 Compare June 25, 2021 09:13

ghost added the Review - Needed The PR is being reviewed label Jul 2, 2021

iSazonov force-pushed the sg-psversion branch from 5abcd36 to 1e6b4cc Compare October 24, 2021 08:36

TravisEz13 reviewed Oct 26, 2021

View reviewed changes

src/SourceGenerators/PSVersionInfoGenerator/PSVersionInfoGenerator.cs Outdated Show resolved Hide resolved

TravisEz13 previously requested changes Oct 26, 2021

View reviewed changes

ghost added Waiting on Author The PR was reviewed and requires changes or comments from the author before being accept and removed Review - Needed The PR is being reviewed labels Oct 26, 2021

TravisEz13 added the WG-Security security related areas such as JEA label Oct 26, 2021

iSazonov force-pushed the sg-psversion branch from 1e6b4cc to 72ad218 Compare October 29, 2021 18:12

ghost removed the Waiting on Author The PR was reviewed and requires changes or comments from the author before being accept label Oct 29, 2021

pull-request-quantifier-deprecated bot added the Small label Oct 29, 2021

Merge part of PowerShell#18210 to the source generator and also refac…

f09bd4d

…tor it

pull-request-quantifier-deprecated bot added Medium and removed Small labels Oct 6, 2022

daxian-dbw reviewed Oct 6, 2022

View reviewed changes

src/System.Management.Automation/System.Management.Automation.csproj Outdated Show resolved Hide resolved

daxian-dbw mentioned this pull request Oct 6, 2022

Make the static constructor of PSVersionInfo run faster for stable PowerShell releases #18210

Merged

13 tasks

Turn off EmitCompilerGeneratedFiles

7a89b2e

- Turn off EmitCompilerGeneratedFiles - Remove first blank line in generated code

iSazonov commented Oct 6, 2022

View reviewed changes

...stem.Management.Automation/SourceGenerators/PSVersionInfoGenerator/PSVersionInfoGenerator.cs Outdated Show resolved Hide resolved

daxian-dbw added 2 commits October 6, 2022 10:08

Rename the function to be 'ParsePSVersion'

44a795e

Make generated files always visible to VSCode and Visual Studio and n…

77ad890

…ot affect re-building

daxian-dbw reviewed Oct 6, 2022

View reviewed changes

Merge branch 'master' into sg-psversion

6998df1

iSazonov commented Oct 7, 2022

View reviewed changes

Address comments

1e26d0c

iSazonov commented Oct 8, 2022

View reviewed changes

daxian-dbw approved these changes Oct 10, 2022

View reviewed changes

daxian-dbw force-pushed the sg-psversion branch from c1a4788 to 1e26d0c Compare October 11, 2022 18:13

Merge branch 'master' into sg-psversion

9cba5cb

daxian-dbw merged commit dd3a6a1 into PowerShell:master Oct 11, 2022

iSazonov deleted the sg-psversion branch October 12, 2022 04:54

kiminuo mentioned this pull request Jul 10, 2023

Add CommitHash to api/software/versions WalletWasabi/WalletWasabi#11003

Merged

Use Source Code Generator for PSVersionInfo class #15603

Use Source Code Generator for PSVersionInfo class #15603

Uh oh!

Conversation

iSazonov commented Jun 18, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Summary

PR Context

PR Checklist

Uh oh!

iSazonov commented Jun 18, 2021

Uh oh!

rjmholt commented Jun 18, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

iSazonov commented Jun 25, 2021

Uh oh!

ghost commented Jul 2, 2021

Uh oh!

Uh oh!

TravisEz13 left a comment

Choose a reason for hiding this comment

Uh oh!

daxian-dbw left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

daxian-dbw Oct 6, 2022

Choose a reason for hiding this comment

Uh oh!

iSazonov Oct 7, 2022

Choose a reason for hiding this comment

Uh oh!

daxian-dbw Oct 7, 2022

Choose a reason for hiding this comment

Uh oh!

iSazonov Oct 7, 2022

Choose a reason for hiding this comment

Uh oh!

daxian-dbw Oct 6, 2022

Choose a reason for hiding this comment

Uh oh!

daxian-dbw commented Oct 6, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

iSazonov left a comment

Choose a reason for hiding this comment

Uh oh!

PaulHigin commented Oct 10, 2022

Uh oh!

daxian-dbw left a comment

Choose a reason for hiding this comment

Uh oh!

pull-request-quantifier-deprecated bot commented Oct 11, 2022

What can I do to optimize my changes

How to interpret the change counts in git diff output

Uh oh!

daxian-dbw commented Oct 11, 2022

Uh oh!

iSazonov commented Oct 12, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

iSazonov commented Oct 12, 2022

Uh oh!

daxian-dbw commented Oct 12, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

iSazonov commented Jun 18, 2021 •

edited

Loading

rjmholt commented Jun 18, 2021 •

edited

Loading

daxian-dbw commented Oct 6, 2022 •

edited

Loading

iSazonov commented Oct 12, 2022 •

edited

Loading

daxian-dbw commented Oct 12, 2022 •

edited

Loading