Skip to content

Implement ForEach-Object -Parallel feature #10229

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
adityapatwardhan merged 27 commits into from
Aug 13, 2019
Merged

Implement ForEach-Object -Parallel feature #10229

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
826ae77
Some refactoring to accomodate parallel processing
PaulHigin Jul 10, 2019
8bd9fd5
Incremental work 1
PaulHigin Jul 12, 2019
88719f3
Work in progress
PaulHigin Jul 15, 2019
12c577f
Completed initial sync implementation
PaulHigin Jul 17, 2019
546fcce
Added basic tests for synchronous mode
PaulHigin Jul 18, 2019
264520b
Add script block errors and more tests
PaulHigin Jul 22, 2019
d49e9de
Add as experimental feature
PaulHigin Jul 22, 2019
4be015f
Add AsJob implementation
PaulHigin Jul 23, 2019
16140c9
Add AsJob tests
PaulHigin Jul 24, 2019
61e5fd9
Code clean up and a couple new tests
PaulHigin Jul 25, 2019
675aea2
Fix test. Move event callback outside lock.
PaulHigin Jul 25, 2019
44e4e7d
Add skip when experimental feature is not enabled
PaulHigin Jul 25, 2019
d26aff3
Code clean up. Fix taskpool Add block to allow data stream writing du…
PaulHigin Jul 26, 2019
17c181b
Initialize dollarUnder variable to correct null value
PaulHigin Jul 29, 2019
2ce7067
Some code clean up
PaulHigin Jul 29, 2019
b87c518
Fix style issues
PaulHigin Jul 29, 2019
fd158e0
Remove ScriptBlock parameter and converted Parallel switch parameter …
PaulHigin Jul 30, 2019
08b3291
Address PR comments
PaulHigin Jul 31, 2019
e434887
Clean up comments based on Codacy and CodeFactor
PaulHigin Aug 5, 2019
1ff1155
More code clean up. Use newer syntax for properties.
PaulHigin Aug 7, 2019
cb72fa7
Response to PR comments
PaulHigin Aug 9, 2019
644eead
Response to PR review comments
PaulHigin Aug 9, 2019
af0cbe9
Add error for unsupported common parameters
PaulHigin Aug 12, 2019
062f5b0
Remove global var conflict with other test
PaulHigin Aug 12, 2019
e31cdf6
Fix CodeFlow comments
PaulHigin Aug 12, 2019
466ba4a
Another attempt to address style issues
PaulHigin Aug 13, 2019
86dd9a5
More style changes
PaulHigin Aug 13, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions src/System.Management.Automation/engine/ExperimentalFeature/ExperimentalFeature.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,9 @@ static ExperimentalFeature()
new ExperimentalFeature(
Copy link
Collaborator

@PoshChan PoshChan Aug 12, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@PaulHigin, your last commit had 1 failures in PowerShell-CI-linux
Get-Variable.Should be able to use wildcard characters in the Name field

Expected 4, but got 'Output:1'.
at <ScriptBlock>, /home/vsts/work/1/s/test/powershell/Modules/Microsoft.PowerShell.Utility/Get-Variable.Tests.ps1: line 80
80: 		(Get-Variable -Name var*).Value[0] | Should -Be 4

name: "PSCommandNotFoundSuggestion",
description: "Recommend potential commands based on fuzzy search on a CommandNotFoundException"),
new ExperimentalFeature(
name: "PSForEachObjectParallel",
description: "New parameter set for ForEach-Object to run script blocks in parallel")
};
EngineExperimentalFeatures = new ReadOnlyCollection<ExperimentalFeature>(engineFeatures);

Expand Down
933 changes: 601 additions & 332 deletions src/System.Management.Automation/engine/InternalCommands.cs
View file
Open in desktop

Large diffs are not rendered by default.

14 changes: 13 additions & 1 deletion src/System.Management.Automation/engine/ScriptCommandProcessor.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -230,13 +230,21 @@ internal override bool IsHelpRequested(out string helpTarget, out HelpCategory h
/// </remarks>
internal sealed class DlrScriptCommandProcessor : ScriptCommandProcessorBase
{
private new ScriptBlock _scriptBlock;
private readonly ArrayList _input = new ArrayList();
private readonly object _dollarUnderbar = AutomationNull.Value;
private new ScriptBlock _scriptBlock;
private MutableTuple _localsTuple;
private bool _runOptimizedCode;
private bool _argsBound;
private FunctionContext _functionContext;

internal DlrScriptCommandProcessor(ScriptBlock scriptBlock, ExecutionContext context, bool useNewScope, CommandOrigin origin, SessionStateInternal sessionState, object dollarUnderbar)
: base(scriptBlock, context, useNewScope, origin, sessionState)
{
Init();
_dollarUnderbar = dollarUnderbar;
}

internal DlrScriptCommandProcessor(ScriptBlock scriptBlock, ExecutionContext context, bool useNewScope, CommandOrigin origin, SessionStateInternal sessionState)
: base(scriptBlock, context, useNewScope, origin, sessionState)
{
Expand Down Expand Up @@ -513,6 +521,10 @@ private void RunClause(Action<FunctionContext> clause, object dollarUnderbar, ob
{
_localsTuple.SetAutomaticVariable(AutomaticVariable.Underbar, dollarUnderbar, _context);
}
else if (_dollarUnderbar != AutomationNull.Value)
{
_localsTuple.SetAutomaticVariable(AutomaticVariable.Underbar, _dollarUnderbar, _context);
}

if (inputToProcess != AutomationNull.Value)
{
Expand Down
10 changes: 9 additions & 1 deletion src/System.Management.Automation/engine/hostifaces/Command.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -173,6 +173,13 @@ internal bool? UseLocalScopeNullable
get { return _useLocalScope; }
}

/// <summary>
/// Gets or sets DollarUnderbar ($_) value to be used with script command.
/// This is used by foreach-object -parallel where each piped input ($_) is associated
/// with a parallel running script block.
/// </summary>
internal object DollarUnderbar { get; set; } = AutomationNull.Value;

/// <summary>
/// Checks if the current command marks the end of a statement (see PowerShell.AddStatement())
/// </summary>
Expand Down Expand Up @@ -495,7 +502,8 @@ CommandOrigin origin
commandProcessorBase = new DlrScriptCommandProcessor(scriptBlock,
executionContext, _useLocalScope ?? false,
origin,
executionContext.EngineSessionState);
executionContext.EngineSessionState,
DollarUnderbar);
}
}
else
Expand Down
Loading