• Is ForEach -Parallel affected in a way?
• In original issue many jobs is created. Make sense to have dedicated thread pool for jobs?

Is ForEach -Parallel affected in a way?

It creates its own thread, so it's not affected.

In original issue many jobs is created. Make sense to have dedicated thread pool for jobs?

I don't think that's needed. Thread pool threads should only for short living tasks. When it comes to long-living tasks, even a separate dedicated thread pool will have the same starvation problem.

Dedicated thread pool for thread jobs is not needed because both ThreadJob and ForEach-Object -Parallel have throttling and queuing, which is basically performs the same function as a thread pool.

@PoshChan Please retry mac

@daxian-dbw, I do not understand the build target(s) mac; I only allow static, windows, macos, linux, all

@PoshChan Please retry macos

@daxian-dbw, successfully started retry of PowerShell-CI-macOS

Turns out the failure in Windows build is a real issue that needs to be looked into. I will investigate and report back.

@daxian-dbw Could you please address CI Windows fail?

@daxian-dbw
I think the issue may be in handling expected errors that reflect various ways listener loop can end. You can see example code for the NamedPipe and SSH transports.

Thanks @PaulHigin for the pointers!
I was busy with some other work with higher priority. Will get back to this soon.

All tests passed now 😄
It turns out I forgot to remove the following two lines from KillServerProcess():

_serverProcess.CancelOutputRead();
_serverProcess.CancelErrorRead();

They should be called only if we are actively using the OutputDataReceived and ErrorDataReceived events to asynchronously read the redirected output and error. Otherwise, they will throw exception which got swollen silently by KillServerProcess() (by design), and thus the line _serverProcess.Kill() will never be executed, which causes the child process started by Start-Job lingers around forever.

The issue results in 34 child pwsh processes not cleaned up after the jobs finish, which eat up the memory in the Windows CI environment. That's why we see the stack overflow and out-of-memory error thrown from CoreCLR (Failed to create CoreCLR, HRESULT: 0x8007000E).

@PaulHigin @iSazonov Can you please take another look at the changes? Thank you!

Sure, I will add a test to verify the server process is terminated after the job is gone.

@PaulHigin, tests are added to verify that the background process is cleaned up after job is gone. Please review it when you have time. Thanks!

@iSazonov I think this is now ready to merge. But please take another look before you do so as there are changes after your last review.

🎉v7.1.0-preview.2 has been released which incorporates this pull request.:tada:

Handy links:

• Release Notes