Skip to content

[pull] main from microsoft:main #166

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
pull merged 3 commits into from
Feb 10, 2026
Merged

[pull] main from microsoft:main #166

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
f3a6dbe
Refactor error handling to extract missing module names from error me…
eleanorjboyd Feb 10, 2026
85d9202
Fix env var warning description (#25758)
anthonykim1 Feb 10, 2026
502ad70
Add support for Projects in Testing (#25780)
eleanorjboyd Feb 10, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/instructions/testing-workflow.instructions.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -578,3 +578,4 @@ envConfig.inspect
- When mocking `testController.createTestItem()` in unit tests, use `typemoq.It.isAny()` for parameters when testing handler behavior (not ID/label generation logic), but consider using specific matchers (e.g., `It.is((id: string) => id.startsWith('_error_'))`) when the actual values being passed are important for correctness - this balances test precision with maintainability (2)
- Remove unused variables from test code immediately - leftover tracking variables like `validationCallCount` that aren't referenced indicate dead code that should be simplified (1)
- Use `Uri.file(path).fsPath` for both sides of path comparisons in tests to ensure cross-platform compatibility - Windows converts forward slashes to backslashes automatically (1)
- When tests fail with "Cannot stub non-existent property", the method likely moved to a different class during refactoring - find the class that owns the method and test that class directly instead of stubbing on the original class (1)
76 changes: 76 additions & 0 deletions .github/instructions/testing_feature_area.instructions.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,10 @@ This document maps the testing support in the extension: discovery, execution (r
- `src/client/testing/serviceRegistry.ts` — DI/wiring for testing services.
- Workspace orchestration
- `src/client/testing/testController/workspaceTestAdapter.ts` — `WorkspaceTestAdapter` (provider-agnostic entry used by controller).
- **Project-based testing (multi-project workspaces)**
- `src/client/testing/testController/common/testProjectRegistry.ts` — `TestProjectRegistry` (manages project lifecycle, discovery, and nested project handling).
- `src/client/testing/testController/common/projectAdapter.ts` — `ProjectAdapter` interface (represents a single Python project with its own test infrastructure).
- `src/client/testing/testController/common/projectUtils.ts` — utilities for project ID generation, display names, and shared adapter creation.
- Provider adapters
- Unittest
- `src/client/testing/testController/unittest/testDiscoveryAdapter.ts`
Expand Down Expand Up @@ -151,6 +155,78 @@ The adapters in the extension don't implement test discovery/run logic themselve
- Settings are consumed by `src/client/testing/common/testConfigurationManager.ts`, `src/client/testing/configurationFactory.ts`, and adapters under `src/client/testing/testController/*` which read settings to build CLI args and env for subprocesses.
- The setting definitions and descriptions are in `package.json` and localized strings in `package.nls.json`.

## Project-based testing (multi-project workspaces)

Project-based testing enables multi-project workspace support where each Python project gets its own test tree root with its own Python environment.

### Architecture

- **TestProjectRegistry** (`testProjectRegistry.ts`): Central registry that:

- Discovers Python projects via the Python Environments API
- Creates and manages `ProjectAdapter` instances per workspace
- Computes nested project relationships and configures ignore lists
- Falls back to "legacy" single-adapter mode when API unavailable

- **ProjectAdapter** (`projectAdapter.ts`): Interface representing a single project with:
- Project identity (ID, name, URI from Python Environments API)
- Python environment with execution details
- Test framework adapters (discovery/execution)
- Nested project ignore paths (for parent projects)

### How it works

1. **Activation**: When the extension activates, `PythonTestController` checks if the Python Environments API is available.
2. **Project discovery**: `TestProjectRegistry.discoverAndRegisterProjects()` queries the API for all Python projects in each workspace.
3. **Nested handling**: `configureNestedProjectIgnores()` identifies child projects and adds their paths to parent projects' ignore lists.
4. **Test discovery**: For each project, the controller calls `project.discoveryAdapter.discoverTests()` with the project's URI. The adapter sets `PROJECT_ROOT_PATH` environment variable for the Python runner.
5. **Python side**:
- For pytest: `get_test_root_path()` in `vscode_pytest/__init__.py` returns `PROJECT_ROOT_PATH` (if set) or falls back to `cwd`.
- For unittest: `discovery.py` uses `PROJECT_ROOT_PATH` as `top_level_dir` and `project_root_path` to root the test tree at the project directory.
6. **Test tree**: Each project gets its own root node in the Test Explorer, with test IDs scoped by project ID using the `@@vsc@@` separator (defined in `projectUtils.ts`).

### Nested project handling: pytest vs unittest

**pytest** supports the `--ignore` flag to exclude paths during test collection. When nested projects are detected, parent projects automatically receive `--ignore` flags for child project paths. This ensures each test appears under exactly one project in the test tree.

**unittest** does not support path exclusion during `discover()`. Therefore, tests in nested project directories may appear under multiple project roots (both the parent and the child project). This is **expected behavior** for unittest:

- Each project discovers and displays all tests it finds within its directory structure
- There is no deduplication or collision detection
- Users may see the same test file under multiple project roots if their project structure has nesting

This approach was chosen because:

1. unittest's `TestLoader.discover()` has no built-in path exclusion mechanism
2. Implementing custom exclusion would add significant complexity with minimal benefit
3. The existing approach is transparent and predictable - each project shows what it finds

### Empty projects and root nodes

If a project discovers zero tests, its root node will still appear in the Test Explorer as an empty folder. This ensures consistent behavior and makes it clear which projects were discovered, even if they have no tests yet.

### Logging prefix

All project-based testing logs use the `[test-by-project]` prefix for easy filtering in the output channel.

### Key files

- Python side:
- `python_files/vscode_pytest/__init__.py` — `get_test_root_path()` function and `PROJECT_ROOT_PATH` environment variable for pytest.
- `python_files/unittestadapter/discovery.py` — `discover_tests()` with `project_root_path` parameter and `PROJECT_ROOT_PATH` handling for unittest discovery.
- `python_files/unittestadapter/execution.py` — `run_tests()` with `project_root_path` parameter and `PROJECT_ROOT_PATH` handling for unittest execution.
- TypeScript: `testProjectRegistry.ts`, `projectAdapter.ts`, `projectUtils.ts`, and the discovery/execution adapters.

### Tests

- `src/test/testing/testController/common/testProjectRegistry.unit.test.ts` — TestProjectRegistry tests
- `src/test/testing/testController/common/projectUtils.unit.test.ts` — Project utility function tests
- `python_files/tests/pytestadapter/test_discovery.py` — pytest PROJECT_ROOT_PATH tests (see `test_project_root_path_env_var()` and `test_symlink_with_project_root_path()`)
- `python_files/tests/unittestadapter/test_discovery.py` — unittest `project_root_path` / PROJECT_ROOT_PATH discovery tests
- `python_files/tests/unittestadapter/test_execution.py` — unittest `project_root_path` / PROJECT_ROOT_PATH execution tests
- `src/test/testing/testController/unittest/testDiscoveryAdapter.unit.test.ts` — unittest discovery adapter PROJECT_ROOT_PATH tests
- `src/test/testing/testController/unittest/testExecutionAdapter.unit.test.ts` — unittest execution adapter PROJECT_ROOT_PATH tests

## Coverage support (how it works)

- Coverage is supported by running the Python helper scripts with coverage enabled and then collecting a coverage payload from the runner.
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/pr-check.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -477,6 +477,8 @@ jobs:
### Coverage run
coverage:
name: Coverage
# TEMPORARILY DISABLED - hanging in CI, needs investigation
if: false
# The value of runs-on is the OS of the current job (specified in the strategy matrix below) instead of being hardcoded.
runs-on: ${{ matrix.os }}
needs: [lint, check-types, python-tests, tests, native-tests]
Expand Down
211 changes: 211 additions & 0 deletions python_files/tests/pytestadapter/expected_discovery_test_output.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1870,3 +1870,214 @@
],
"id_": TEST_DATA_PATH_STR,
}

# =====================================================================================
# PROJECT_ROOT_PATH environment variable tests
# These test the project-based testing feature where PROJECT_ROOT_PATH changes
# the test tree root from cwd to the specified project path.
# =====================================================================================

# This is the expected output for unittest_folder when PROJECT_ROOT_PATH is set to unittest_folder.
# The root of the tree is unittest_folder (not .data), simulating project-based testing.
#
# **Project Configuration:**
# In the VS Code Python extension, projects are defined by the Python Environments extension.
# Each project has a root directory (identified by pyproject.toml, setup.py, etc.).
# When PROJECT_ROOT_PATH is set, pytest uses that path as the test tree root instead of cwd.
#
# **Test Tree Structure:**
# Without PROJECT_ROOT_PATH (legacy mode):
# └── .data (cwd = workspace root)
# └── unittest_folder
# └── test_add.py, test_subtract.py...
#
# With PROJECT_ROOT_PATH set to unittest_folder (project-based mode):
# └── unittest_folder (ROOT - set via PROJECT_ROOT_PATH env var)
# ├── test_add.py
# │ └── TestAddFunction
# │ ├── test_add_negative_numbers
# │ └── test_add_positive_numbers
# │ └── TestDuplicateFunction
# │ └── test_dup_a
# └── test_subtract.py
# └── TestSubtractFunction
# ├── test_subtract_negative_numbers
# └── test_subtract_positive_numbers
# └── TestDuplicateFunction
# └── test_dup_s
#
# Note: This reuses the unittest_folder paths defined earlier in this file.
project_root_unittest_folder_expected_output = {
"name": "unittest_folder",
"path": os.fspath(unittest_folder_path),
"type_": "folder",
"children": [
{
"name": "test_add.py",
"path": os.fspath(test_add_path),
"type_": "file",
"id_": os.fspath(test_add_path),
"children": [
{
"name": "TestAddFunction",
"path": os.fspath(test_add_path),
"type_": "class",
"children": [
{
"name": "test_add_negative_numbers",
"path": os.fspath(test_add_path),
"lineno": find_test_line_number(
"test_add_negative_numbers",
os.fspath(test_add_path),
),
"type_": "test",
"id_": get_absolute_test_id(
"test_add.py::TestAddFunction::test_add_negative_numbers",
test_add_path,
),
"runID": get_absolute_test_id(
"test_add.py::TestAddFunction::test_add_negative_numbers",
test_add_path,
),
},
{
"name": "test_add_positive_numbers",
"path": os.fspath(test_add_path),
"lineno": find_test_line_number(
"test_add_positive_numbers",
os.fspath(test_add_path),
),
"type_": "test",
"id_": get_absolute_test_id(
"test_add.py::TestAddFunction::test_add_positive_numbers",
test_add_path,
),
"runID": get_absolute_test_id(
"test_add.py::TestAddFunction::test_add_positive_numbers",
test_add_path,
),
},
],
"id_": get_absolute_test_id(
"test_add.py::TestAddFunction",
test_add_path,
),
"lineno": find_class_line_number("TestAddFunction", test_add_path),
},
{
"name": "TestDuplicateFunction",
"path": os.fspath(test_add_path),
"type_": "class",
"children": [
{
"name": "test_dup_a",
"path": os.fspath(test_add_path),
"lineno": find_test_line_number(
"test_dup_a",
os.fspath(test_add_path),
),
"type_": "test",
"id_": get_absolute_test_id(
"test_add.py::TestDuplicateFunction::test_dup_a",
test_add_path,
),
"runID": get_absolute_test_id(
"test_add.py::TestDuplicateFunction::test_dup_a",
test_add_path,
),
},
],
"id_": get_absolute_test_id(
"test_add.py::TestDuplicateFunction",
test_add_path,
),
"lineno": find_class_line_number("TestDuplicateFunction", test_add_path),
},
],
},
{
"name": "test_subtract.py",
"path": os.fspath(test_subtract_path),
"type_": "file",
"id_": os.fspath(test_subtract_path),
"children": [
{
"name": "TestSubtractFunction",
"path": os.fspath(test_subtract_path),
"type_": "class",
"children": [
{
"name": "test_subtract_negative_numbers",
"path": os.fspath(test_subtract_path),
"lineno": find_test_line_number(
"test_subtract_negative_numbers",
os.fspath(test_subtract_path),
),
"type_": "test",
"id_": get_absolute_test_id(
"test_subtract.py::TestSubtractFunction::test_subtract_negative_numbers",
test_subtract_path,
),
"runID": get_absolute_test_id(
"test_subtract.py::TestSubtractFunction::test_subtract_negative_numbers",
test_subtract_path,
),
},
{
"name": "test_subtract_positive_numbers",
"path": os.fspath(test_subtract_path),
"lineno": find_test_line_number(
"test_subtract_positive_numbers",
os.fspath(test_subtract_path),
),
"type_": "test",
"id_": get_absolute_test_id(
"test_subtract.py::TestSubtractFunction::test_subtract_positive_numbers",
test_subtract_path,
),
"runID": get_absolute_test_id(
"test_subtract.py::TestSubtractFunction::test_subtract_positive_numbers",
test_subtract_path,
),
},
],
"id_": get_absolute_test_id(
"test_subtract.py::TestSubtractFunction",
test_subtract_path,
),
"lineno": find_class_line_number("TestSubtractFunction", test_subtract_path),
},
{
"name": "TestDuplicateFunction",
"path": os.fspath(test_subtract_path),
"type_": "class",
"children": [
{
"name": "test_dup_s",
"path": os.fspath(test_subtract_path),
"lineno": find_test_line_number(
"test_dup_s",
os.fspath(test_subtract_path),
),
"type_": "test",
"id_": get_absolute_test_id(
"test_subtract.py::TestDuplicateFunction::test_dup_s",
test_subtract_path,
),
"runID": get_absolute_test_id(
"test_subtract.py::TestDuplicateFunction::test_dup_s",
test_subtract_path,
),
},
],
"id_": get_absolute_test_id(
"test_subtract.py::TestDuplicateFunction",
test_subtract_path,
),
"lineno": find_class_line_number("TestDuplicateFunction", test_subtract_path),
},
],
},
],
"id_": os.fspath(unittest_folder_path),
}
Loading
Loading