2

I would like to implement a -parallel Switch to one of my skripts

Non Parallel Version:

  $tmpArray | ForEach-Object {
          #region ... local Variables
          $stepWidthLocal = $stepWidth
<#
my code
#>

Parallel Funktion:

  $tmpArray | ForEach-Object -ThrottleLimit ($parallelBlocks * 10) -Parallel {
          #region ... local Variables
          $stepWidthLocal = $using:stepWidth
<#
my code
#>

What I dont want is:

$myParallel = $true
if ($myParallel) {
  $tmpArray | ForEach-Object -ThrottleLimit ($parallelBlocks * 10) -Parallel {
    #region ... local Variables
    $stepWidthLocal = $using:stepWidth
    <#
    my code
    #>
  } #end $tmpArray | ForEach-Object -ThrottleLimit ($parallelBlocks * 10) -Parallel
} #end if($myParallel) 
else {
  $tmpArray | ForEach-Object {
    #region ... local Variables
    $stepWidthLocal = $stepWidth
    <#
my code
#>
  } #end $tmpArray | ForEach-Object {
} #end else {

I want something like this:


$myCode = <#
define my Codeblock
#>
$myParallel = $true
if ($myParallel) {
  $tmpArray | ForEach-Object -ThrottleLimit ($parallelBlocks * 10) -Parallel {
    #region ... local Variables
    $stepWidthLocal = $using:stepWidth
    $myCode
  } #end $tmpArray | ForEach-Object -ThrottleLimit ($parallelBlocks * 10) -Parallel
} #end if($myParallel) 
else {
  $tmpArray | ForEach-Object {
    #region ... local Variables
    $stepWidthLocal = $stepWidth
    $myCode
  } #end $tmpArray | ForEach-Object {
} #end else {

Now I want to create some kind of switch statement without duplicating the entire code (block <# my code#>).

is this possible?

0

2 Answers 2

1

You can define the reusable code as a script block, but note that you won't be able to use it directly in your ForEach-Object -Parallel script block and instead have to recreate it there, via its string representation passed to the static [scriptblock]::Create() method; using a simplified example:

# Your reusable code block.
# Note the .ToString() call to obtain its string representation.
$myCodeSource = {
  "hi from thread $([System.Threading.Thread]::CurrentThread.ManagedThreadId)"
}.ToString()


1, 2 | ForEach-Object -Parallel { 
  # ...
  # Note: You can pass arguments, if the script block is prepared to handle them.
  & ([scriptblock]::Create($using:myCodeSource)) 
}

Note: This answer contains an analogous solution for using a function from the caller's scope in a ForEach-Object -Parallel script block.

The above output something like the following:

hi from thread 35
hi from thread 36

Note (as of PowerShell 7.2):

  • ForEach-Object -Parallel actively prevents direct use of script blocks from the caller's scope (accessed via the $using:) scope, because using script blocks across threads can lead to thread-safety issues; curiously, however, the related Start-ThreadJob does accept script blocks via $using: - though that may have been an oversight.

    • Recreating a script block via its string representation, as shown above, works around this limitation.
  • GitHub issue #12378 discusses this behavior, including a possible enhancement to let ForEach-Object itself automatically recreate the script block in a thread-safe manner.

    • More generally, GitHub issue #12240 proposes an opt-in mechanism that would allow copying the caller's definitions to each thread, making $using: references unnecessary.
Sign up to request clarification or add additional context in comments.

1 Comment

Glad to hear it, @user3898488; my pleasure.
1
$arguments = @{ }
if ($myParallel) {
    $arguments = @{ parallel = $true; $throttlelimit = 10 }
}
$tmpArray | ForEach-Object @arguments {
...
}

1 Comment

I tested this solution but I failed as I cannot pass the variables to the block. You need to pass the variables with $using: to the parallelblock: devblogs.microsoft.com/powershell/… which doesnt work in the nonparellel block. So you could implement a $useParallel switch but as you cannot pass the $useParallel itself, this doesnt work either. So for all I know, the idea is great, but it doesnt work

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.