Add `--backend` flag to over-ride manual detection of uv by nathanjmcdougall · Pull Request #826 · usethis-python/usethis-python

nathanjmcdougall · 2025-06-29T06:58:20Z

This involves some major architectural changes.

Fix add_* function when needing to remove Tweak test config

codecov · 2025-06-30T05:48:26Z

❌ 44 Tests Failed:

Tests completed	Failed	Passed	Skipped
1298	44	1254	6

View the top 3 failed test(s) by shortest run time

tests/usethis/_tool/impl/test_coverage_py.py::TestCoveragePyTool::TestAddConfigs::test_after_codespell

Stack Traces | 0.022s run time

self = <tests.usethis._tool.impl.test_coverage_py.TestCoveragePyTool.TestAddConfigs object at 0x7f50f7e13750>
tmp_path = PosixPath('.../pytest-of-runner/pytest-0/test_after_codespell0')

            def test_after_codespell(self, tmp_path: Path):
                # To check the config is valid
                # https://github..../usethis-python/issues/558
    
                # Arrange
                (tmp_path / "pyproject.toml").write_text("""\
    [project]
    name = "example"
    version = "0.1.0"
    description = "Add your description here"
    
    [dependency-groups]
    dev = [
        "codespell>=2.4.1",
    ]
    
    [tool.codespell]
    ignore-regex = ["[A-Za-z0-9+/]{100,}"]
    """)
    
                # Act
                with change_cwd(tmp_path), files_manager():
                    CoveragePyTool().add_configs()
    
                # Assert
                with change_cwd(tmp_path), files_manager():
                    assert ["tool", "coverage"] in PyprojectTOMLManager()
>               assert "[tool.coverage]" in (tmp_path / "pyproject.toml").read_text()
E               assert '[tool.coverage]' in '[project]\nname = "example"\nversion = "0.1.0"\ndescription = "Add your description here"\n\n[dependency-groups]\ndev = [\n    "codespell>=2.4.1",\n]\n                                                    \n[tool.codespell]\nignore-regex = ["[A-Za-z0-9+/]{100,}"]\n\n[tool.coverage.run]\nsource = ["."]\n'
E                +  where '[project]\nname = "example"\nversion = "0.1.0"\ndescription = "Add your description here"\n\n[dependency-groups]\ndev = [\n    "codespell>=2.4.1",\n]\n                                                    \n[tool.codespell]\nignore-regex = ["[A-Za-z0-9+/]{100,}"]\n\n[tool.coverage.run]\nsource = ["."]\n' = read_text()
E                +    where read_text = (PosixPath('.../pytest-of-runner/pytest-0/test_after_codespell0') / 'pyproject.toml').read_text

self       = <tests.usethis._tool.impl.test_coverage_py.TestCoveragePyTool.TestAddConfigs object at 0x7f50f7e13750>
tmp_path   = PosixPath('.../pytest-of-runner/pytest-0/test_after_codespell0')

.../_tool/impl/test_coverage_py.py:38: AssertionError

tests/usethis/_core/test_core_tool.py::TestCodespell::TestAdd::test_codespell_rc_file[NetworkConn.OFFLINE]

Stack Traces | 0.046s run time

self = <tests.usethis._core.test_core_tool.TestCodespell.TestAdd object at 0x7f50f7aa4140>
uv_init_dir = PosixPath('.../pytest-of-runner/pytest-0/test_codespell_rc_file_Network1')

            @pytest.mark.usefixtures("_vary_network_conn")
            def test_codespell_rc_file(self, uv_init_dir: Path):
                # This file is only preferred to pyproject.toml if there's already
                # some codespell config in the file
    
                # Arrange
                (uv_init_dir / ".codespellrc").write_text(
                    """\
    [codespell]
    fake = bar
    """
                )
    
                # Act
                with change_cwd(uv_init_dir), files_manager():
                    use_codespell()
    
                # Assert
>               assert (uv_init_dir / ".codespellrc").read_text() == (
                    """\
    [codespell]
    fake = bar
    ignore-regex = [A-Za-z0-9+/]{100,}
    ignore-words-list = ...
    """
                )
E               AssertionError: assert '[codespell]\...-9+/]{100,}\n' == '[codespell]\...-list = ...\n'
E                 
E                   [codespell]
E                   fake = bar
E                   ignore-regex = [A-Za-z0-9+/]{100,}
E                 - ignore-words-list = ...

self       = <tests.usethis._core.test_core_tool.TestCodespell.TestAdd object at 0x7f50f7aa4140>
uv_init_dir = PosixPath('.../pytest-of-runner/pytest-0/test_codespell_rc_file_Network1')

.../usethis/_core/test_core_tool.py:172: AssertionError

tests/usethis/_core/test_core_tool.py::TestCodespell::TestAdd::test_codespell_rc_file[NetworkConn.ONLINE]

Stack Traces | 0.047s run time

self = <tests.usethis._core.test_core_tool.TestCodespell.TestAdd object at 0x7f50f7a94250>
uv_init_dir = PosixPath('.../pytest-of-runner/pytest-0/test_codespell_rc_file_Network0')

            @pytest.mark.usefixtures("_vary_network_conn")
            def test_codespell_rc_file(self, uv_init_dir: Path):
                # This file is only preferred to pyproject.toml if there's already
                # some codespell config in the file
    
                # Arrange
                (uv_init_dir / ".codespellrc").write_text(
                    """\
    [codespell]
    fake = bar
    """
                )
    
                # Act
                with change_cwd(uv_init_dir), files_manager():
                    use_codespell()
    
                # Assert
>               assert (uv_init_dir / ".codespellrc").read_text() == (
                    """\
    [codespell]
    fake = bar
    ignore-regex = [A-Za-z0-9+/]{100,}
    ignore-words-list = ...
    """
                )
E               AssertionError: assert '[codespell]\...-9+/]{100,}\n' == '[codespell]\...-list = ...\n'
E                 
E                   [codespell]
E                   fake = bar
E                   ignore-regex = [A-Za-z0-9+/]{100,}
E                 - ignore-words-list = ...

self       = <tests.usethis._core.test_core_tool.TestCodespell.TestAdd object at 0x7f50f7a94250>
uv_init_dir = PosixPath('.../pytest-of-runner/pytest-0/test_codespell_rc_file_Network0')

.../usethis/_core/test_core_tool.py:172: AssertionError

To view more test analytics, go to the Test Analytics Dashboard
_{📋 Got 3 mins? Take this short survey to help us improve Test Analytics.}

codspeed-hq · 2025-06-30T05:49:53Z

CodSpeed Instrumentation Performance Report

Merging #826 will degrade performances by 8.46%

_{Comparing 241-flag-to-avoid-calling-uv (e3aa6e5) with main (5d0b818)}

Summary

❌ 1 (👁 1) regressions
🆕 1 new benchmarks
⁉️ 1 (👁 1) dropped benchmarks

Benchmarks breakdown

	Benchmark	`BASE`	`HEAD`	Change
👁	`test_help_flag`	39.2 ms	42.9 ms	-8.46%
👁	`test_several_tools_add_and_remove`	1.4 s	N/A	N/A
🆕	`test_several_tools_add_and_remove`	N/A	1.5 s	N/A

nathanjmcdougall · 2025-07-12T04:31:53Z

There are still a couple of major things to do here:

There's still lots of calls to call_uv_subprocess which aren't protected by backend dispatching
Testing the higher-level interfaces with a none backend

nathanjmcdougall · 2025-07-12T21:10:48Z

Another thing to do - we shouldn't create pyproject.toml anymore unless it's truly necessary. With the none backend, we aren't adding deps in the pyproject.toml file (or anywhere), just displaying a message, so it shouldn't be necessary to create this file.

nathanjmcdougall · 2025-07-13T22:19:31Z

And yet another thing - make sure all the --how / print_how_to_use messages are respecting the backend choice

nathanjmcdougall · 2025-07-23T11:19:51Z

Now testing all interfaces which support the --backend option.

Need to consider whether the usethis ci commands should respect backend choice internally to the CI config. I think they need to, in which case that needs to be implemented and tested.

nathanjmcdougall · 2025-07-23T11:55:23Z

Blocked by #663, since we are assuming empty files are valid here:

usethis-python/src/usethis/_tool/base.py

Lines 417 to 421 in 2982eb3

    
           for file_manager in file_managers: 
        
               if not (file_manager.path.exists() and file_manager.path.is_file()): 
        
                   # If the file doesn't exist, we will create it 
        
                   file_manager.path.touch(exist_ok=True)

We should test that empty files are valid for all key-value file managers, probably via a constant that stores all of them (#905)

Also, I'd like to break up the complexity here so we don't need to disable the Ruff rule:

usethis-python/src/usethis/_tool/base.py

Lines 382 to 383 in 2982eb3

    
           def add_configs(self) -> None:  # noqa: PLR0912 
        
               """Add the tool's configuration sections.

…ol.apply_rule_config` This is something to watch out for in the future.

nathanjmcdougall · 2025-07-25T04:53:23Z

I think making the CI config backend agnostic is an important next step, but it really belongs in a separate PR. At the moment, the CI completely assumes uv backend. And the same goes for pre-commit config per #273

nathanjmcdougall · 2025-07-25T05:22:31Z

Hitting #912 and some tests are out of date based on the new #908 behaviour

nathanjmcdougall · 2025-07-29T10:29:55Z

#912 should just be resolved here in this branch because I think managed-file creation is a new functionality/default behaviour (should check this).

usethis-python/src/usethis/_tool/base.py

Lines 430 to 434 in b6c88b6

    
           for file_manager in used_file_managers: 
        
               if not (file_manager.path.exists() and file_manager.path.is_file()): 
        
                   # If the file doesn't exist, we will create it 
        
                   file_manager.path.touch(exist_ok=True)

Pass all `test_none_backend` tests

nathanjmcdougall · 2025-07-29T11:02:47Z

I think everything is ready except to make sure all the --how / print_how_to_use messages are respecting the backend choice, and in general checking invocations of is_uv_used to determine whether they're better replaced by a backend-based logic

Copilot

Pull Request Overview

This PR adds a --backend flag to override manual detection of UV, representing a major architectural change to support multiple package manager backends. The changes introduce a backend enum system that allows usethis to work with or without UV, providing better flexibility for users who may not have UV installed.

Key changes:

Introduction of a backend system with BackendEnum (auto, uv, none) to control package manager behavior
Major reorganization of the codebase structure, moving interface modules and creating backend dispatch logic
Addition of --backend flag across all CLI commands to allow manual backend selection

Reviewed Changes

Copilot reviewed 109 out of 121 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
`src/usethis/_types/backend.py`	Defines the new BackendEnum for backend selection
`src/usethis/_integrations/backend/dispatch.py`	Implements backend detection and selection logic
`src/usethis/_ui/options.py`	Adds the backend option for CLI commands
`src/usethis/_config.py`	Updates configuration to include backend settings
`src/usethis/_deps.py`	Refactors dependency management to work with multiple backends
`src/usethis/_init.py`	Creates unified project initialization logic
Multiple test files	Updates tests to work with the new backend system

Comments suppressed due to low confidence (2)

src/usethis/_types/deps.py:14

The method name to_requirement_string is inconsistent with the method call to_requirements_string used in the old code. This creates an inconsistency that could cause issues.

    def to_requirement_string(self) -> str:

tests/usethis/_types/test_deps.py:8

The test uses to_requirement_string() which matches the new implementation, but this inconsistency with the old to_requirements_string() method name should be addressed for consistency.

            assert dep.to_requirement_string() == "requests"

src/usethis/_integrations/backend/uv/used.py

src/usethis/_tool/base.py

src/usethis/_tool/impl/ruff.py

src/usethis/_init.py

First draft of adding --backend flag to the usethis tool interface

41761ff

nathanjmcdougall linked an issue Jun 29, 2025 that may be closed by this pull request

Flag to avoid calling uv #241

Closed

nathanjmcdougall added 4 commits June 30, 2025 17:17

Tweak backend selection logic to check whether uv is available.

d11eb0c

Fix add_* function when needing to remove Tweak test config

Move tests to reflect new package structure

75f78a1

Remove duplicated Test in TestOpinionatedUVInit name

80ceaf2

Revert to importing module to pass mocked tests

3f853e5

nathanjmcdougall added 6 commits June 30, 2025 18:29

Add tests for is_uv_available and for get_backend

b6a10ad

Move tests module to reflect package restructure

01a6d15

Test add_default_groups for the backend=BackendEnum.none case

151d45a

Merge branch 'main' into 241-flag-to-avoid-calling-uv

e538536

Remove empty file

7efdca7

Add some tests to the deps module

8075131

nathanjmcdougall marked this pull request as ready for review July 12, 2025 04:06

nathanjmcdougall requested a review from Copilot July 12, 2025 04:06

This comment was marked as outdated.

Sign in to view

Document ForbiddenBackendError in docstring to call_uv_subprocess

aa25d60

nathanjmcdougall added 5 commits July 12, 2025 16:33

Add --backend option to all interfaces where it makes sense

013bcee

Fix broken test

47583a4

Remove disclaimers regarding uv-centrism

d9612b6

Merge branch 'main' into 241-flag-to-avoid-calling-uv

d844616

Merge branch 'main' into 241-flag-to-avoid-calling-uv

f968dd4

nathanjmcdougall changed the title ~~Add --backend flag to over-ride manual detection of uv~~ Add --backend flag to over-ride manual detection of uv Jul 12, 2025

nathanjmcdougall added 3 commits July 13, 2025 07:35

Bump pyright from v1.1.399 -> v1.1.403

bc5c591

Merge branch 'maint/bump-pyright' into 241-flag-to-avoid-calling-uv

ff4e343

Add some tests for the none backend when using requirements.txt

31bec7f

Fix backend flag for usethis status

084e5c4

nathanjmcdougall added 2 commits July 23, 2025 23:24

Don't create README in use_ci_bitbucket

cfc903a

Don't eagerly create pyproject.toml in usethis tool codespell

2982eb3

nathanjmcdougall linked an issue Jul 25, 2025 that may be closed by this pull request

Handle case where uv is not installed #21

Closed

nathanjmcdougall added 2 commits July 25, 2025 16:15

Break up the complexity of Tool.add_configs with a new helper method

8b84ca3

Don't overwrite an alert-only state in Tool.add_configs and `RuffTo…

b6c88b6

…ol.apply_rule_config` This is something to watch out for in the future.

nathanjmcdougall mentioned this pull request Jul 25, 2025

Include changes to pyproject.toml creation in next changelog #908

Closed

nathanjmcdougall mentioned this pull request Jul 25, 2025

Make bitbucket CI config backend agnostic #909

Closed

nathanjmcdougall removed a link to an issue Jul 25, 2025

Handle case where uv is not installed #21

Closed

This was linked to issues Jul 25, 2025

Internal config to avoid subprocessing uv #296

Closed

Don't define tool.uv.default-groups unless uv is being used #535

Closed

nathanjmcdougall mentioned this pull request Jul 29, 2025

Emit message when creating a config file when adding config items #912

Closed

Display message when creating input files when adding config

7797773

Pass all `test_none_backend` tests

nathanjmcdougall added 4 commits July 30, 2025 06:06

Merge branch 'main' into 241-flag-to-avoid-calling-uv

eb94f3a

Pass test_none_backend tests

f698950

Change test to reflect new behaviour

12ba8a9

Respect backend in print-how-to-use messages

4c46aee

nathanjmcdougall requested a review from Copilot August 1, 2025 06:23

Copilot AI reviewed Aug 1, 2025

View reviewed changes

src/usethis/_integrations/backend/uv/used.py Outdated Show resolved Hide resolved

src/usethis/_tool/base.py Outdated Show resolved Hide resolved

src/usethis/_tool/impl/ruff.py Outdated Show resolved Hide resolved

src/usethis/_init.py Show resolved Hide resolved

nathanjmcdougall added 3 commits August 1, 2025 18:33

Don't use bitwise operators to avoid confusion

94cddf6

rework is_uv_used for robustness

609e670

Refactor add_configs for clarity

e3aa6e5

nathanjmcdougall merged commit f44669a into main Aug 3, 2025
19 checks passed

nathanjmcdougall deleted the 241-flag-to-avoid-calling-uv branch August 3, 2025 23:26

Conversation

nathanjmcdougall commented Jun 29, 2025

Uh oh!

codecov bot commented Jun 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

❌ 44 Tests Failed:

Uh oh!

codspeed-hq bot commented Jun 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

CodSpeed Instrumentation Performance Report

Merging #826 will degrade performances by 8.46%

Summary

Benchmarks breakdown

Uh oh!

This comment was marked as outdated.

Uh oh!

nathanjmcdougall commented Jul 12, 2025

Uh oh!

nathanjmcdougall commented Jul 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nathanjmcdougall commented Jul 13, 2025

Uh oh!

nathanjmcdougall commented Jul 23, 2025

Uh oh!

nathanjmcdougall commented Jul 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nathanjmcdougall commented Jul 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nathanjmcdougall commented Jul 25, 2025

Uh oh!

nathanjmcdougall commented Jul 29, 2025

Uh oh!

nathanjmcdougall commented Jul 29, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

codecov bot commented Jun 30, 2025 •

edited

Loading

codspeed-hq bot commented Jun 30, 2025 •

edited

Loading

nathanjmcdougall commented Jul 12, 2025 •

edited

Loading

nathanjmcdougall commented Jul 23, 2025 •

edited

Loading

nathanjmcdougall commented Jul 25, 2025 •

edited

Loading