Skip to content

Fix for #6855 - Measure-Object should handle scriptblock properties. - #6934

Merged
Dongbo Wang (daxian-dbw) merged 7 commits into
PowerShell:masterfrom
BrucePay:brucepay_MeasureObject
Jun 20, 2018
Merged

Dongbo Wang (daxian-dbw) merged 7 commits into
PowerShell:masterfrom
BrucePay:brucepay_MeasureObject

Conversation

@BrucePay

@BrucePay Bruce Payette (BrucePay) commented May 25, 2018 •

Copy link
Copy Markdown
Collaborator

PR Summary

Fixed issue #6855 by renaming the class that does all the work from MshExpression to PSPropertyExpression, making the class public and then in MeasureObjectCommand, lifting it up to the parameter level. Previously the implementation exposed the -Property as string then wrapped it internally as a PSPropertyExpression. Now the parameter type is PSPropertyExpression directly allowing for both wildcard strings and scriptblocks. Note that this is technically a breaking change (the public type of the parameter has changed) but this was reviewed by the committee and approved. (The vast majority of changes in this PR are due to the renaming. The code changes are relatively small.)

PSPropertyExpression now lives in a public namespace where it can be used by cmdlet and script authors to easily add the same type of functionality to their commands.

I also modified PSPropertyExpression to treat hashtables as objects so that

@{prop = 3} | measure-object prop

and

@{prop = 3} | measure-object {$_.prop}

will work the identically. (Previously the example using just the property name would just fail.)

Breaking Change Summary

This PR brings in a breaking change in an edge case:

This could be a breaking change when it comes to the ConstrainedLanguage mode.
@{prop = 3} | measure-object Count works in ConstrainedLanguage mode today, but with this change, the conversion from Hashtable to PSCustomObject will throw exception.

It has been reviewed by the committee and agreed to accept this breaking change.
Please see the discussion at #6934 (comment) for more information.

PR Checklist

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we use pipeline make sense use some values:

$result = "a,b,c", "d,e" | Measure-Object -Word  {$_ -split ','}
$result.Words | Should -Be 5

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe move the function and tests in new Context block?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved into a separate Describe.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add empty new line.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added.

@iSazonov

Copy link
Copy Markdown
Collaborator

We should get PowerShell Committee conclusion for new public API.

@BrucePay

Copy link
Copy Markdown
Collaborator Author

Ilya (@iSazonov) We reviewed this in the last committee meeting and approved the name. I thought Steve Lee (@SteveL-MSFT) had updated the bug to reflect this but I don't see it.

@iSazonov

Copy link
Copy Markdown
Collaborator

Bruce Payette (@BrucePay) Thanks! I am extecting that the PR will marked by the label.
Seems we lost all MSFT team in recent weeks. I wonder.

@BrucePay

Copy link
Copy Markdown
Collaborator Author

Oops - I had the wrong issue number in the comment so you might have looked at the wrong issue. #6855 is, in fact, marked "Committee Reviewed".

@iSazonov Ilya (iSazonov) left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with one minor comment.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please use C# 7 syntax?

if (PSObject.Base(target) as Hashtable targetAsHash) 

@iSazonov Ilya (iSazonov) added the Documentation Needed in this repo Documentation is needed in this repo label Jun 1, 2018
@BrucePay

Copy link
Copy Markdown
Collaborator Author

Steve Lee (@SteveL-MSFT) Says we're ignoring CodeFactor issues for now.

@SteveL-MSFT

Copy link
Copy Markdown
Member

Ilya (@iSazonov) FYI - Most of my team has been involved with helping port existing Windows PowerShell modules owned by other teams to work with PSCore6 so you may find us not as active. Hopefully we'll be back to fully engaged by end of June (or mostly engaged).

@iSazonov

Copy link
Copy Markdown
Collaborator

I'm glad that this process began. I do not see this leading to corrections in PowerShell Core - it convinces me that PowerShell Core is very good! 😄

@daxian-dbw Dongbo Wang (daxian-dbw) left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/cc Paul Higinbotham (@PaulHigin) Now that PSPropertyExpression is publically exposed and it has the ability to run scripts, Paul you may want to check if this will introduce any security concerns to the ConstrainedLanguage mode.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could be a breaking change when it comes to the ConstrainedLanguage mode.
@{prop = 3} | measure-object Count works in ConstrainedLanguage mode today, but with this change, the conversion from Hashtable to PSCustomObject will throw exception.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm - true - good catch. It's pretty obscure and currently the only useful scenario for piping hashtables into measure-object (other than instance-counting). Still, we should probably get a committee review on this.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please change the name to WrapHashtableAsPSCustomObject? C# method naming conversion suggests using verbs or verb phrases to name methods.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The point of the rule is that the names of methods should be imperative which this is. There are other examples of (type 1) conditional clauses in .NET such as ThrowIfCancellationRequested on System.Threading.CancellationToken.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK. #Closed

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we want to handle hashtable specially, shall we expand that to IDictionary? LanguagePrimitives.ConvertPSObjectToType is able to convert IDictionary to PSObject.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The goal is to match the behaviour of [PSCustomObject] @{a=1; b=2}. Currently casting Dictionary<T,T> to PSCustomObject doesn't result in a custom object so I don't think we should introduce that behaviour here.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Understood. Good to know that we are matching the behavior of [PSCustomObject] @{a=1; b=2}.

@BrucePay Bruce Payette (BrucePay) added the Review - Committee The PR/Issue needs a review from the PowerShell Committee label Jun 8, 2018

@daxian-dbw Dongbo Wang (daxian-dbw) left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Waiting on the committee's review.

@PaulHigin

Copy link
Copy Markdown
Contributor

Dongbo Wang (@daxian-dbw) Bruce Payette (@BrucePay) Sorry for the late review. To answer Dongbo's question, from a security point of view I think this is Ok. MshExpression (now PSPropertyExpression) takes a ScriptBlock in its constructor which it will invoke. But as long as the passed in ScriptBlock is created in a constrained environment it will be marked accordingly and the script block will be invoked with CL mode constraints.

A constructor also takes a string "expression", but as near as I can tell it is never evaluated as script and is used to look up properties. Is this correct?

However, we should include a CL mode test to ensure the expression scriptblock is run in CL mode.

@daxian-dbw

Copy link
Copy Markdown
Member

A constructor also takes a string "expression", but as near as I can tell it is never evaluated as script and is used to look up properties. Is this correct?

Paul Higinbotham (@PaulHigin) Yes, that's correct. It's for property lookup.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Paul Higinbotham (@PaulHigin) I would like you to take another look at this change. PSPropertyExpression is added to CoreTypes, and that means users can call its methods in ConstrainedLanguage mode. Do you have any concerns about it?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I didn't see that. If this type is going to be white listed then it needs to go through a separate/full security review. I see that some internal methods are now public, and these along with constructors need to be reviewed for vulnerabilities. Please set up a meeting for this.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No - it's only intended to be a type accelerator. I added it in the wrong place. I'll move it to the TypeAccelerators() method where it should be.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I agree it should be moved :)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I didn't see that. If this type is going to be white listed then it needs to go through a separate/full security review. I see that some internal methods are now public, and these along with constructors need to be reviewed for vulnerabilities. Please set up a meeting for this.

@SteveL-MSFT Steve Lee (SteveL-MSFT) added Committee-Reviewed PS-Committee has reviewed this and made a decision and removed Review - Committee The PR/Issue needs a review from the PowerShell Committee labels Jun 14, 2018
@SteveL-MSFT

Copy link
Copy Markdown
Member

@PowerShell/powershell-committee reviewed this and is fine with this breaking change which is likely only an edge case, but the positive impact for most users is beneficial

@SteveL-MSFT Steve Lee (SteveL-MSFT) added the Breaking-Change breaking change that may affect users label Jun 14, 2018
Bruce Payette added 6 commits June 15, 2018 17:10
…operties. Fixed by renaming MshExpression to PSPropertyExpression and making it

public. Then in MeasureObjectCommand, lifting it up to the parameter level. Previously the implementation exposed the Property as string and
wrapped it internally as a PSPropertyExpression. Now the parameter type is PSPropertyExpression directly allowing for both wildcard strings
and scriptblocks. PSPropertyExpression now lives in a public namespace where it can be used by cmdlet and script authors to easily add the
same type of functionality to their commands. I also modified PSPropertyExpression to handle hashtables properly as objects so
@{prop = 3} | measure-object prop
and
@{prop = 3} | measure-object {$_.prop}
will work the same. (Previously the example using just the property name would fail.)
Added missing type accelerator test, cleaned up tests.
@daxian-dbw

Copy link
Copy Markdown
Member

Paul Higinbotham (@PaulHigin) New commits have been pushed. Can you please update your review? Thanks

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@daxian-dbw

Copy link
Copy Markdown
Member

Bruce Payette (@BrucePay) I just noticed that we didn't run feature tests for this PR CI. Can you please push another commit with [feature] in the commit message? That will trigger the feature test run. You can also update the current head commit by running git commit --amend in your local brucepay_MeasureObject branch, and then forcing push to the remote by git push --force-with-lease <your fork repo> brucepay_MeasureObject. Thanks!

@daxian-dbw

Copy link
Copy Markdown
Member

Bruce Payette (@BrucePay) I have submitted a dummy commit to trigger the feature test run for this PR.

@daxian-dbw
Dongbo Wang (daxian-dbw) merged commit 3a4c410 into PowerShell:master Jun 20, 2018
@joeyaiello Joey Aiello (joeyaiello) removed Documentation Needed in this repo Documentation is needed in this repo labels Oct 15, 2018
Mikey Lombardi (He/Him) (michaeltlombardi) added a commit to michaeltlombardi/PowerShellDocs that referenced this pull request Oct 17, 2022
Prior to this change, the documentation for `Group-Object`,
`Select-Object`, and `Sort-Object` did not specifically address using
those cmdlets against **hashtable** input.

In PowerShell 6 an improvement (see PowerShell/PowerShell#6934) enabled
users to reference the values of **hashtable** keys in these commands
without having to specify calculated properties.

This change:

- Updates the documentation for `Group-Object`, `Select-Object`, and
  `Sort-Object` in versions 7.0 and above to show how users can group,
  select, and sort **hashtable** objects by their key values
- Updates the documentation for `Group-Object`, `Select-Object`, and
  `Sort-Object` in Windows PowerShell 5.1 to show how users can group,
  select, and sort **hashtable** objects with calculated properties
- Cleans up Vale linting warnings across those files
- Fixes AB#20552
- Resolves MicrosoftDocs#9311
Sean Wheeler (sdwheeler) pushed a commit to michaeltlombardi/PowerShellDocs that referenced this pull request Oct 17, 2022
Prior to this change, the documentation for `Group-Object`,
`Select-Object`, and `Sort-Object` did not specifically address using
those cmdlets against **hashtable** input.

In PowerShell 6 an improvement (see PowerShell/PowerShell#6934) enabled
users to reference the values of **hashtable** keys in these commands
without having to specify calculated properties.

This change:

- Updates the documentation for `Group-Object`, `Select-Object`, and
  `Sort-Object` in versions 7.0 and above to show how users can group,
  select, and sort **hashtable** objects by their key values
- Updates the documentation for `Group-Object`, `Select-Object`, and
  `Sort-Object` in Windows PowerShell 5.1 to show how users can group,
  select, and sort **hashtable** objects with calculated properties
- Cleans up Vale linting warnings across those files
- Fixes AB#20552
- Resolves MicrosoftDocs#9311
Sean Wheeler (sdwheeler) pushed a commit to MicrosoftDocs/PowerShell-Docs that referenced this pull request Oct 17, 2022
Prior to this change, the documentation for `Group-Object`,
`Select-Object`, and `Sort-Object` did not specifically address using
those cmdlets against **hashtable** input.

In PowerShell 6 an improvement (see PowerShell/PowerShell#6934) enabled
users to reference the values of **hashtable** keys in these commands
without having to specify calculated properties.

This change:

- Updates the documentation for `Group-Object`, `Select-Object`, and
  `Sort-Object` in versions 7.0 and above to show how users can group,
  select, and sort **hashtable** objects by their key values
- Updates the documentation for `Group-Object`, `Select-Object`, and
  `Sort-Object` in Windows PowerShell 5.1 to show how users can group,
  select, and sort **hashtable** objects with calculated properties
- Cleans up Vale linting warnings across those files
- Fixes AB#20552
- Resolves #9311
Thatgfsj (Thatgfsj) pushed a commit to Thatgfsj-contribs/PowerShell that referenced this pull request Aug 6, 2026
…operties. (PowerShell#6934)

Measure-Object should handle `ScriptBlock` properties. Fixed by renaming `MshExpression` to `PSPropertyExpression` and making it public. Then in `MeasureObjectCommand`, lifting it up to the parameter level. Previously the implementation exposed the Property as a string and
wrapped it internally as a `PSPropertyExpression`. Now the parameter type is `PSPropertyExpression` directly allowing for both wildcard strings and `ScriptBlock`.

`PSPropertyExpression` now lives in a public namespace where it can be used by cmdlet and script authors to easily add the same type of functionality to their commands. I also modified `PSPropertyExpression` to handle hashtables properly as objects so
         @{prop = 3} | measure-object prop
and
         @{prop = 3} | measure-object {$_.prop}
will work the same. (Previously the example using just the property name would fail.)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Breaking-Change breaking change that may affect users Committee-Reviewed PS-Committee has reviewed this and made a decision

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants