feat: add @pytest.mark.benchmark to all tests #308

shenxianpeng · 2025-10-13T20:01:10Z

Summary by CodeRabbit

Tests
- Introduced benchmarking markers across many existing tests to collect performance metrics during test runs.
- Applies to configuration handling, import fallbacks, engine behavior, rule building, utilities, and main execution paths.
- Purely additive annotations; assertions and control flow remain unchanged.
- No changes to product behavior, settings, or user-facing features.
- Improves visibility into performance trends and helps detect regressions over time.

sonarqubecloud · 2025-10-13T20:01:31Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

coderabbitai · 2025-10-13T20:01:46Z

Walkthrough

Added pytest import statements where needed and applied @pytest.mark.benchmark decorators to existing tests across multiple test files. No test logic, control flow, or assertions were changed; only benchmark metadata was introduced.

Changes

Cohort / File(s)	Summary
Config tests `tests/config_edge_test.py`, `tests/config_fallback_test.py`, `tests/config_import_test.py`, `tests/config_test.py`	Added pytest import where missing and annotated selected tests with `@pytest.mark.benchmark`; no logic changes.
Engine tests `tests/engine_comprehensive_test.py`, `tests/engine_test.py`	Applied `@pytest.mark.benchmark` to numerous tests; imported pytest as needed; no functional changes.
Main tests `tests/main_test.py`	Decorated multiple tests with `@pytest.mark.benchmark`; no changes to assertions or flow.
Rule builder tests `tests/rule_builder_test.py`	Added pytest import and benchmark decorators to several tests; no behavioral changes.
Util tests `tests/util_test.py`	Marked selected tests with `@pytest.mark.benchmark`; no modifications to test logic or APIs.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Poem

A rabbit taps keys with benchmarking cheer,
Tagging each test so the timings appear.
No code paths shifted, no branches askew—
Just swifter insights, hop-hop, through and through.
Stopwatch ears perked, I nibble and grin:
“Mark, set, benchmark!” Let the runs begin. 🐇⏱️

Pre-merge checks and finishing touches

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title Check	✅ Passed	The title succinctly describes the main change by indicating that the pytest benchmark marker is being added to all tests and follows conventional commit style, making it both clear and specific to the changeset.
Docstring Coverage	✅ Passed	Docstring coverage is 94.81% which is sufficient. The required threshold is 80.00%.

✨ Finishing touches

📝 Generate docstrings

🧪 Generate unit tests (beta)

Create PR with unit tests
Post copyable unit tests in a comment
Commit unit tests in branch feature/add-pytest.mark.benchmark

📜 Recent review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between d0bf5e7 and cfa4f5b.

📒 Files selected for processing (9)

tests/config_edge_test.py (3 hunks)
tests/config_fallback_test.py (1 hunks)
tests/config_import_test.py (2 hunks)
tests/config_test.py (8 hunks)
tests/engine_comprehensive_test.py (16 hunks)
tests/engine_test.py (62 hunks)
tests/main_test.py (21 hunks)
tests/rule_builder_test.py (9 hunks)
tests/util_test.py (5 hunks)

🧰 Additional context used

📓 Path-based instructions (1)

tests/**

📄 CodeRabbit inference engine (.github/copilot-instructions.md)

Ensure tests run via pytest -v and cover commit, branch, author, and CLI behaviors

Files:

tests/engine_comprehensive_test.py
tests/config_edge_test.py
tests/util_test.py
tests/engine_test.py
tests/rule_builder_test.py
tests/config_import_test.py
tests/main_test.py
tests/config_fallback_test.py
tests/config_test.py

🧠 Learnings (1)

📚 Learning: 2025-10-03T10:28:06.793Z

Learnt from: CR
PR: commit-check/commit-check#0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-10-03T10:28:06.793Z
Learning: Applies to commit_check/author.py : Validate author email format

Applied to files:

tests/engine_test.py

🧬 Code graph analysis (3)

tests/engine_comprehensive_test.py (2)

tests/engine_test.py (2)

TestValidationResult (26-31)

TestValidationContext (34-49)

commit_check/engine.py (1)

ValidationResult (19-23)

tests/engine_test.py (2)

commit_check/engine.py (2)

ValidationResult (19-23)

BaseValidator (35-94)

tests/engine_comprehensive_test.py (2)

TestValidationContext (31-43)

TestCommitMessageValidator (46-116)

tests/main_test.py (1)

commit_check/main.py (1)

main (127-225)

🪛 Ruff (0.14.0)

tests/main_test.py

27-27: Unused method argument: capfd

(ARG002)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (11)

GitHub Check: install (3.12, windows-latest)
GitHub Check: install (3.14, macos-latest)
GitHub Check: install (3.13, macos-latest)
GitHub Check: install (3.13, windows-latest)
GitHub Check: install (3.14, windows-latest)
GitHub Check: install (3.10, windows-latest)
GitHub Check: install (3.10, macos-latest)
GitHub Check: install (3.11, windows-latest)
GitHub Check: install (3.9, windows-latest)
GitHub Check: install (3.9, macos-latest)
GitHub Check: Run benchmarks

🔇 Additional comments (9)

tests/config_test.py (1)

12-12: LGTM! Benchmark decorators properly applied.

The @pytest.mark.benchmark decorators are correctly added to all test methods without modifying test logic.

Also applies to: 32-32, 41-41, 61-61, 81-81, 94-94, 109-109, 116-116, 139-139

tests/main_test.py (1)

11-11: LGTM! Benchmark decorators properly applied across all test methods.

The decorators are correctly placed and do not modify any test logic.

Also applies to: 19-19, 26-26, 32-32, 42-42, 55-55, 68-68, 82-82, 96-96, 110-110, 124-124, 134-134, 148-148, 157-157, 169-169, 184-184, 195-195, 211-211, 225-225, 235-235, 249-249, 267-267, 290-290, 303-303

tests/config_edge_test.py (1)

9-9: LGTM! Benchmark decorators correctly applied.

All three test functions are properly decorated with @pytest.mark.benchmark.

Also applies to: 27-27, 50-50

tests/engine_test.py (1)

27-27: LGTM! Comprehensive benchmark decoration across all test methods.

All test methods in this large test file have been properly decorated with @pytest.mark.benchmark without any test logic modifications.

Also applies to: 35-35, 44-44, 53-53, 61-61, 74-74, 87-87, 105-105, 123-123, 145-145, 163-163, 179-179, 193-193, 203-203, 217-217, 231-231, 256-256, 267-267, 279-279, 292-292, 304-304, 319-319, 329-329, 339-339, 356-356, 374-374, 391-391, 409-409, 428-428, 438-438, 450-450, 460-460, 472-472, 486-486, 496-496, 508-508, 519-519, 533-533, 543-543, 555-555, 567-567, 577-577, 587-587, 599-599, 613-613, 628-628, 646-646, 657-657, 670-670, 682-682, 692-692, 705-705, 714-714, 725-725, 740-740, 758-758, 768-768, 781-781, 794-794, 811-811, 821-821, 833-833, 844-844, 855-855

tests/config_fallback_test.py (1)

6-6: LGTM! pytest import and benchmark decorator properly added.

The required pytest import is added and the test function is correctly decorated.

Also applies to: 10-10

tests/engine_comprehensive_test.py (1)

20-20: LGTM! pytest import and benchmark decorators properly added.

The pytest import is added and all test methods are correctly decorated with @pytest.mark.benchmark.

Also applies to: 24-24, 32-32, 47-47, 60-60, 79-79, 101-101, 120-120, 136-136, 156-156, 172-172, 192-192, 208-208, 228-228, 242-242, 270-270, 288-288, 306-306

tests/rule_builder_test.py (1)

5-5: LGTM! pytest import and benchmark decorators properly added.

The pytest import is added and all test methods are correctly decorated.

Also applies to: 9-9, 19-19, 33-33, 45-45, 64-64, 78-78, 99-99, 120-120, 142-142

tests/config_import_test.py (2)

6-6: LGTM!

The pytest import is correctly added to support the decorator usage.

9-9: No changes needed for @pytest.mark.benchmark: The benchmark marker is registered as a no-op in pyproject.toml under tool.pytest.ini_options, so it’s intentionally used for grouping tests.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

codecov · 2025-10-13T20:02:39Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87.29%. Comparing base (d0bf5e7) to head (cfa4f5b).
⚠️ Report is 5 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #308   +/-   ##
=======================================
  Coverage   87.29%   87.29%           
=======================================
  Files           8        8           
  Lines         685      685           
=======================================
  Hits          598      598           
  Misses         87       87

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

codspeed-hq · 2025-10-13T20:05:51Z

CodSpeed Performance Report

Merging #308 will not alter performance

_{Comparing feature/add-pytest.mark.benchmark (cfa4f5b) with main (d0bf5e7)}

Summary

✅ 27 untouched
🆕 129 new
⏩ 85 skipped¹

Benchmarks breakdown

	Benchmark	`BASE`	`HEAD`	Change
🆕	`test_load_config_file_permission_error`	N/A	331 µs	N/A
🆕	`test_load_config_invalid_toml`	N/A	409 µs	N/A
🆕	`test_tomli_import_fallback`	N/A	414.6 µs	N/A
🆕	`test_config_tomli_fallback_direct`	N/A	1.3 ms	N/A
🆕	`test_import_paths_coverage`	N/A	317 µs	N/A
🆕	`test_tomli_import_fallback_simulation`	N/A	1.2 ms	N/A
🆕	`test_default_config_paths_constant`	N/A	180.2 µs	N/A
🆕	`test_load_config_default_cchk_toml`	N/A	360.9 µs	N/A
🆕	`test_load_config_default_commit_check_toml`	N/A	356.4 µs	N/A
🆕	`test_load_config_file_not_found`	N/A	243.2 µs	N/A
🆕	`test_load_config_file_not_found_with_invalid_path_hint`	N/A	322.1 µs	N/A
🆕	`test_load_config_with_nonexistent_path_hint`	N/A	226.6 µs	N/A
🆕	`test_load_config_with_path_hint`	N/A	383.9 µs	N/A
🆕	`test_toml_load_function_exists`	N/A	329.3 µs	N/A
🆕	`test_tomli_import_fallback`	N/A	1.4 ms	N/A
🆕	`test_commit_message_validator_creation`	N/A	118.5 µs	N/A
🆕	`test_commit_message_validator_failure`	N/A	2 ms	N/A
🆕	`test_commit_message_validator_skip_validation`	N/A	912.5 µs	N/A
🆕	`test_commit_message_validator_with_stdin`	N/A	938 µs	N/A
🆕	`test_subject_capitalization_fail`	N/A	1.1 ms	N/A
...	...	...	...	...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

85 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

feat: add @pytest.mark.benchmark to all tests

cfa4f5b

shenxianpeng requested a review from a team as a code owner October 13, 2025 20:01

shenxianpeng added the enhancement New feature or request label Oct 13, 2025

shenxianpeng merged commit bfcc722 into main Oct 13, 2025
32 checks passed

shenxianpeng deleted the feature/add-pytest.mark.benchmark branch October 13, 2025 20:20

shenxianpeng added tests Add test related changes and removed enhancement New feature or request labels Oct 20, 2025

coderabbitai bot mentioned this pull request Dec 14, 2025

feat: Add comprehensive tests #331

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

feat: add @pytest.mark.benchmark to all tests #308

feat: add @pytest.mark.benchmark to all tests #308

Uh oh!

shenxianpeng commented Oct 13, 2025 •

edited by coderabbitai bot

Loading

Uh oh!

sonarqubecloud bot commented Oct 13, 2025

Uh oh!

coderabbitai bot commented Oct 13, 2025 •

edited

Loading

Uh oh!

codecov bot commented Oct 13, 2025 •

edited

Loading

Uh oh!

codspeed-hq bot commented Oct 13, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

feat: add @pytest.mark.benchmark to all tests #308

feat: add @pytest.mark.benchmark to all tests #308

Uh oh!

Conversation

shenxianpeng commented Oct 13, 2025 • edited by coderabbitai bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by CodeRabbit

Uh oh!

sonarqubecloud bot commented Oct 13, 2025

Quality Gate passed

Uh oh!

coderabbitai bot commented Oct 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Estimated code review effort

Poem

Pre-merge checks and finishing touches

Uh oh!

codecov bot commented Oct 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

codspeed-hq bot commented Oct 13, 2025

CodSpeed Performance Report

Merging #308 will not alter performance

Summary

Benchmarks breakdown

Footnotes

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

shenxianpeng commented Oct 13, 2025 •

edited by coderabbitai bot

Loading

coderabbitai bot commented Oct 13, 2025 •

edited

Loading

codecov bot commented Oct 13, 2025 •

edited

Loading