Skip to content

Move xUnit tests in new folder - #8356

Merged
Travis Plunk (TravisEz13) merged 10 commits into
PowerShell:masterfrom
iSazonov:move-xunit-tests2
Dec 13, 2018
Merged

Travis Plunk (TravisEz13) merged 10 commits into
PowerShell:masterfrom
iSazonov:move-xunit-tests2

Conversation

@iSazonov

@iSazonov Ilya (iSazonov) commented Nov 29, 2018 •

Copy link
Copy Markdown
Collaborator

(Replace #8237)

Motivation

I have a PR where there are many new xUnit tests.
It would also be useful to create new xUnit tests for public APIs.
The number of xUnit tests will increase and their ordering is required.

PR Summary

  • Move C# xUnit tests in new folder. This allows to put new xUnit tests in directory structure in accordance with directory structure where cs files are.
  • Use an xUnit TestCaseOrderer attribute to sequentially process tests for powershell.config.json.
  • Update README.md
  • A race condition was fixed which allowed to run all XUnit tests in single batch job.

PR Checklist

Remove unneeded dotnet restore

Add  --no-restore

Add extraParams for MacOS

Fix TestRunspaceWithPowerShellAndInitialSessionState

Fix race condition to access powershell.config.json

Reduce an impact of the inter process race condition

Fix race condition to access powershell.config.json (PowerShell#8249)
@iSazonov

Copy link
Copy Markdown
Collaborator Author

Dongbo Wang (@daxian-dbw) Travis Plunk (@TravisEz13) Aditya Patwardhan (@adityapatwardhan) Could you please review the PR? It blocks my follow PRs (one could be used to speed up startup).

@@ -0,0 +1,45 @@
using System;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add copyright header.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

@@ -0,0 +1,12 @@
using System;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add copyright header.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still missing copyright header here.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I lost new commits.

Fixed.

Comment thread build.psm1
}

# we are having intermittent issues on macOS with these tests failing.
# VSTS has suggested forcing them to be sequential

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we validate if this is still the case. If not, we can removed the condition.

@iSazonov Ilya (iSazonov) Nov 30, 2018 •

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps Travis Plunk (@TravisEz13) can comment this.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We would have to try it to know.

@iSazonov Ilya (iSazonov) Dec 1, 2018 •

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could open a tracking issue for the work and don't block the PR.

Also we have many CodeFactor issues. Also we need to have more tests for binary APIs. Also we need to have performance tests. I think we should open new tracking issues.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done #8400

public void TestRunspaceWithPowerShellAndInitialSessionState()
{
InitialSessionState iss = InitialSessionState.CreateDefault2();
InitialSessionState iss = InitialSessionState.CreateDefault();

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why was this changed?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two tests above use the CreateDefault(). I guess it was typo. If we need tests for CreateDefault2() I believe it should be added in new PR.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe CreateDefault2 is intentional as it creates an ISS without any modules loaded. Please change it back.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reverted with new comment.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aditya Patwardhan (@adityapatwardhan) that seems like a method name that probably needs revising, if we can, or adding an alias for in some fashion. Ilya is generally pretty familiar with the repo; if he got it confused, it's likely others will also.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changing the methodName would be a huge breaking change, but we could add a new method that calls this that is named better. I would suggest filing an issue about this.

@adityapatwardhan

Copy link
Copy Markdown
Member

Ilya (@iSazonov) Please resolve the merge conflict.

Comment thread test/xUnit/xUnit.tests.csproj Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why was this added?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was temporary additions.

Removed.

@iSazonov

Ilya (iSazonov) commented Dec 6, 2018 •

Copy link
Copy Markdown
Collaborator Author

I restarted CI-Windows - there was a race condition again. Perhaps it is only test specific issue. Aditya Patwardhan (@adityapatwardhan) Dongbo Wang (@daxian-dbw) could you please look? There test cleanup code failed but I don't understand that process lock the config file. I guess it is other test processes. In this case we need improve the test cleanup code like we did in WaitFile().

@iSazonov

Copy link
Copy Markdown
Collaborator Author

I updated #8400 to fix the rare race condition in tests.

Aditya Patwardhan (@adityapatwardhan) We could merge.

@iSazonov

Copy link
Copy Markdown
Collaborator Author

Aditya Patwardhan (@adityapatwardhan) Could you please update your code review and merge?

@TravisEz13

Copy link
Copy Markdown
Member

Ilya (@iSazonov) Aditya Patwardhan (@adityapatwardhan) Is out of the office until the new year.

@iSazonov

Copy link
Copy Markdown
Collaborator Author

Travis Plunk (@TravisEz13) If you could merge I could continue my work.

Remove unneeded dotnet restore

Add  --no-restore

Add extraParams for MacOS

Fix TestRunspaceWithPowerShellAndInitialSessionState

Fix race condition to access powershell.config.json

Reduce an impact of the inter process race condition

Fix race condition to access powershell.config.json (PowerShell#8249)
@TravisEz13

Copy link
Copy Markdown
Member

rebased the branch mainly to verify the tests still pass

@iSazonov

Copy link
Copy Markdown
Collaborator Author

The test TestRunspaceWithPowerShellAndInitialSessionState fail exclusively and randomly on MacOs if we use CreateDefault2(). The test passes with CreateDefault(). I guess that previously we hided an race condition issue on MacOs by using sequential tests.
I added the issue to #8400 for tracking the issue and added commit to use ``CreateDefault()`.

@TravisEz13
Travis Plunk (TravisEz13) merged commit a9f106d into PowerShell:master Dec 13, 2018
@iSazonov
Ilya (iSazonov) deleted the move-xunit-tests2 branch December 14, 2018 06:24
Thatgfsj (Thatgfsj) pushed a commit to Thatgfsj-contribs/PowerShell that referenced this pull request Aug 6, 2026
## Motivation

I have a PR where there are many new xUnit tests.
It would also be useful to create new xUnit tests for public APIs.
The number of xUnit tests will increase and their ordering is required.

## PR Summary

- Move C# xUnit tests in new folder. This allows to put new xUnit tests in directory structure in accordance with directory structure where cs files are.
- Use an xUnit TestCaseOrderer attribute to sequentially process tests for `powershell.config.json`.
- Update README.md
- A race condition was fixed which allowed  to run all XUnit tests in single batch job.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CL-Test Indicates that a PR should be marked as a test change in the Change Log

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants