Skip to content

Use Source Code Generator for PSVersionInfo class - #15603

Merged
Dongbo Wang (daxian-dbw) merged 15 commits into
PowerShell:masterfrom
iSazonov:sg-psversion
Oct 11, 2022
Merged

Dongbo Wang (daxian-dbw) merged 15 commits into
PowerShell:masterfrom
iSazonov:sg-psversion

Conversation

@iSazonov

@iSazonov Ilya (iSazonov) commented Jun 18, 2021 •

Copy link
Copy Markdown
Collaborator

PR Summary

The custom source generator allows us to exclude reflection and to directly get current PowerShell version from MSBuild.

PerfView doesn't work well on current .Net Preview. A fix has just been merged in .Net Runtime repo for this. I hope we get it in .Net 6.0 Preview6 or Preview7. So today I can not share reliable result from PerfView to confirm perf win. Obviously using static string is more faster then reflection.

PR Context

Contribute to #13540
Related #14268

PR Checklist

@iSazonov

Copy link
Copy Markdown
Collaborator Author

It seems test fails are not related to the PR - pwsh.exe has Preview.8 in product version instead of Preview.7.

Rob Holt (@rjmholt) Travis Plunk (@TravisEz13) Could you please look the issue?

@rjmholt

Rob Holt (rjmholt) commented Jun 18, 2021 •

Copy link
Copy Markdown
Collaborator

Ilya (@iSazonov) I opened #15607

@iSazonov
Ilya (iSazonov) force-pushed the sg-psversion branch 4 times, most recently from cbb19d0 to 5abcd36 Compare June 25, 2021 09:13
@iSazonov

Copy link
Copy Markdown
Collaborator Author

Rob Holt (@rjmholt) I resolved the issue with release tag. It turned out that ci.psm1 always assigns ReleaseTag property directly to dotnet.

@ghost ghost added the Review - Needed The PR is being reviewed label Jul 2, 2021
@ghost

Copy link
Copy Markdown

This pull request has been automatically marked as Review Needed because it has been there has not been any activity for 7 days.
Maintainer, please provide feedback and/or mark it as Waiting on Author

@TravisEz13 Travis Plunk (TravisEz13) left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pending @wg-security review

@ghost ghost added Waiting on Author The PR was reviewed and requires changes or comments from the author before being accept and removed Review - Needed The PR is being reviewed labels Oct 26, 2021
@TravisEz13 Travis Plunk (TravisEz13) added the WG-Security security related areas such as JEA label Oct 26, 2021
@ghost ghost removed the Waiting on Author The PR was reviewed and requires changes or comments from the author before being accept label Oct 29, 2021

@daxian-dbw Dongbo Wang (daxian-dbw) left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I integrated the FastParsePSVersion change from #18210 to the source generator, and also refactored the code. After the change, the static constructor of PSVersionTable only takes 2-3 ms on my dev machine (release build with R2R images, using the release tag v7.3.0 to mimic a stable release). It was 24-30 ms without the changes of this PR:

## Measured by wrapping the static constructor with `StopWatch`.

C:\>E:\arena\source\PowerShell\src\powershell-win-core\bin\Release\net7.0\win7-x64\publish\pwsh.exe -noprofile
2.5478
PowerShell 7.3.0
PS C:\> exit

C:\>E:\arena\source\PowerShell\src\powershell-win-core\bin\Release\net7.0\win7-x64\publish\pwsh.exe -noprofile
2.5993
PowerShell 7.3.0
PS C:\> exit

C:\>E:\arena\source\PowerShell\src\powershell-win-core\bin\Release\net7.0\win7-x64\publish\pwsh.exe -noprofile
2.5842
PowerShell 7.3.0
PS C:\> exit

- Turn off EmitCompilerGeneratedFiles
- Remove first blank line in generated code
<!-- we persist source generator files under 'gen' folder so that they are visible to IDEs -->
<EmitCompilerGeneratedFiles>true</EmitCompilerGeneratedFiles>
<CompilerGeneratedFilesOutputPath>gen\SourceGenerated</CompilerGeneratedFilesOutputPath>
</PropertyGroup>

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generate the file, so VSCode and Visual Studio can always resolve the generated members.
Those files will be removed when building again, by RemoveDir in PowerShell.Common.props.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I pushed a commit a day before (but you reverted now) and removed this old workaround because all works well in VS Code. We have no need to emit generated files.
Perhaps you need re-configure your local environment.
I use VS Code 1.72 and C# extension 1.25
image

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You didn't see my reply to your comment? It's here: #15603 (comment), and quoted below:

It doesn't work in the following scenarios, for both VSCode and Visual Studio, which are quite common in development:

  1. Run Start-PSBuild first, then open System.Management.Automation in VSCode
  2. Open System.Management.Automation in VSCode to use an existing build, like from yesterday.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, too many comments already....

Comment thread PowerShell.Common.props
</PropertyGroup>

<!-- Remove existing generated files by source generators because new source code will soon be generated when the build starts -->
<RemoveDir Directories="gen\SourceGenerated" Condition="Exists('gen\SourceGenerated')" />

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The path gen\SourceGenerated is duplicated in here and System.Management.Automation.csproj. I tried creating a property holding the path in the PropertyGroup right above, and use it in RemoveDir and CompilerGeneratedFilesOutputPath. However, it works in RemoveDir, but not in CompilerGeneratedFilesOutputPath, and I cannot figure out why. I'd love if someone can make it work.

@daxian-dbw
Dongbo Wang (daxian-dbw) dismissed Travis Plunk (TravisEz13)’s stale review October 6, 2022 21:03

The review was done a long time ago. Will request to re-review.

@daxian-dbw

Dongbo Wang (daxian-dbw) commented Oct 6, 2022 •

Copy link
Copy Markdown
Member

Travis Plunk (@TravisEz13) and Paul Higinbotham (@PaulHigin), can you please review the changes again and re-evaluate the "code injection" concern?

Here is the doc (and the sample code) about the source generator: https://learn.microsoft.com/en-us/dotnet/csharp/roslyn-sdk/source-generators-overview

The arguments used to format the source template in PSVersionInfoGenerator.cs are from the build properties defined in PowerShell.Common.props.

@iSazonov Ilya (iSazonov) left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@PaulHigin

Copy link
Copy Markdown
Contributor

@WG-Security
The security group is Ok with this, because for release builds the generated version comes from a trusted source (releaseTag).

@daxian-dbw Dongbo Wang (daxian-dbw) left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@pull-request-quantifier-deprecated

Copy link
Copy Markdown

This PR has 115 quantified lines of changes. In general, a change size of upto 200 lines is ideal for the best PR experience!


Quantification details

Label      : Medium
Size       : +98 -17
Percentile : 43%

Total files changed: 5

Change summary by file extension:
.props : +2 -0
.cs : +62 -16
.csproj : +34 -1

Change counts above are quantified counts, based on the PullRequestQuantifier customizations.

Why proper sizing of changes matters

Optimal pull request sizes drive a better predictable PR flow as they strike a
balance between between PR complexity and PR review overhead. PRs within the
optimal size (typical small, or medium sized PRs) mean:

  • Fast and predictable releases to production:
    • Optimal size changes are more likely to be reviewed faster with fewer
      iterations.
    • Similarity in low PR complexity drives similar review times.
  • Review quality is likely higher as complexity is lower:
    • Bugs are more likely to be detected.
    • Code inconsistencies are more likely to be detected.
  • Knowledge sharing is improved within the participants:
    • Small portions can be assimilated better.
  • Better engineering practices are exercised:
    • Solving big problems by dividing them in well contained, smaller problems.
    • Exercising separation of concerns within the code changes.

What can I do to optimize my changes

  • Use the PullRequestQuantifier to quantify your PR accurately
    • Create a context profile for your repo using the context generator
    • Exclude files that are not necessary to be reviewed or do not increase the review complexity. Example: Autogenerated code, docs, project IDE setting files, binaries, etc. Check out the Excluded section from your prquantifier.yaml context profile.
    • Understand your typical change complexity, drive towards the desired complexity by adjusting the label mapping in your prquantifier.yaml context profile.
    • Only use the labels that matter to you, see context specification to customize your prquantifier.yaml context profile.
  • Change your engineering behaviors
    • For PRs that fall outside of the desired spectrum, review the details and check if:
      • Your PR could be split in smaller, self-contained PRs instead
      • Your PR only solves one particular issue. (For example, don't refactor and code new features in the same PR).

How to interpret the change counts in git diff output

  • One line was added: +1 -0
  • One line was deleted: +0 -1
  • One line was modified: +1 -1 (git diff doesn't know about modified, it will
    interpret that line like one addition plus one deletion)
  • Change percentiles: Change characteristics (addition, deletion, modification)
    of this PR in relation to all other PRs within the repository.


Was this comment helpful? 👍  :ok_hand:  :thumbsdown: (Email)
Customize PullRequestQuantifier for this repository.

@daxian-dbw

Copy link
Copy Markdown
Member

Ilya (@iSazonov) Just FYI, I re-did the conflict resolution because you accidentally brough back the PSCompatibleVersions property, which was deleted in the other PR.

@daxian-dbw
Dongbo Wang (daxian-dbw) merged commit dd3a6a1 into PowerShell:master Oct 11, 2022
@iSazonov
Ilya (iSazonov) deleted the sg-psversion branch October 12, 2022 04:54
@iSazonov

Ilya (iSazonov) commented Oct 12, 2022 •

Copy link
Copy Markdown
Collaborator Author

Dongbo Wang (@daxian-dbw) Thanks for working on the SG!

Since you mentioned saving build time I think you could be interesting in SG for TypeGen.

@iSazonov

Copy link
Copy Markdown
Collaborator Author

Ilya (@iSazonov) Just FYI, I re-did the conflict resolution because you accidentally brough back the PSCompatibleVersions property, which was deleted in the other PR.

Gihub did show only one merge conflict for

       internal static string ProductVersion { get; }
        internal static string GitCommitId
        {
            get
            {
                return (string)s_psVersionTable[PSGitCommitIdName];
            }
        }

but did not marked PSCompatibleVersions. I wonder why. 😕

@daxian-dbw

Dongbo Wang (daxian-dbw) commented Oct 12, 2022 •

Copy link
Copy Markdown
Member

Since you mentioned saving build time I think you could be interesting in SG for TypeGen.

I don't know if that would save build time, I doubt it :)
Again, it's not about saving build time, but more about not doing things that are not needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CL-Performance Indicates that a PR should be marked as a performance improvement in the Change Log Medium WG-Security security related areas such as JEA

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants