Skip to content

feat: add @pytest.mark.benchmark to all tests#308

Merged
shenxianpeng merged 1 commit intomainfrom
feature/add-pytest.mark.benchmark
Oct 13, 2025
Merged

feat: add @pytest.mark.benchmark to all tests#308
shenxianpeng merged 1 commit intomainfrom
feature/add-pytest.mark.benchmark

Conversation

@shenxianpeng
Copy link
Contributor

@shenxianpeng shenxianpeng commented Oct 13, 2025 •
edited by coderabbitai bot
Loading

Summary by CodeRabbit

  • Tests
    • Introduced benchmarking markers across many existing tests to collect performance metrics during test runs.
    • Applies to configuration handling, import fallbacks, engine behavior, rule building, utilities, and main execution paths.
    • Purely additive annotations; assertions and control flow remain unchanged.
    • No changes to product behavior, settings, or user-facing features.
    • Improves visibility into performance trends and helps detect regressions over time.

@shenxianpeng shenxianpeng requested a review from a team as a code owner October 13, 2025 20:01
@shenxianpeng shenxianpeng added the enhancement New feature or request label Oct 13, 2025
@sonarqubecloud
Copy link

sonarqubecloud bot commented Oct 13, 2025

Quality Gate Passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Oct 13, 2025 •
edited
Loading

Walkthrough

Added pytest import statements where needed and applied @pytest.mark.benchmark decorators to existing tests across multiple test files. No test logic, control flow, or assertions were changed; only benchmark metadata was introduced.

Changes

Cohort / File(s) Summary
Config tests
tests/config_edge_test.py, tests/config_fallback_test.py, tests/config_import_test.py, tests/config_test.py		 Added pytest import where missing and annotated selected tests with @pytest.mark.benchmark; no logic changes.
Engine tests
tests/engine_comprehensive_test.py, tests/engine_test.py		 Applied @pytest.mark.benchmark to numerous tests; imported pytest as needed; no functional changes.
Main tests
tests/main_test.py		 Decorated multiple tests with @pytest.mark.benchmark; no changes to assertions or flow.
Rule builder tests
tests/rule_builder_test.py		 Added pytest import and benchmark decorators to several tests; no behavioral changes.
Util tests
tests/util_test.py		 Marked selected tests with @pytest.mark.benchmark; no modifications to test logic or APIs.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Poem

A rabbit taps keys with benchmarking cheer,
Tagging each test so the timings appear.
No code paths shifted, no branches askew—
Just swifter insights, hop-hop, through and through.
Stopwatch ears perked, I nibble and grin:
“Mark, set, benchmark!” Let the runs begin. 🐇⏱️

Pre-merge checks and finishing touches

✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title Check ✅ Passed The title succinctly describes the main change by indicating that the pytest benchmark marker is being added to all tests and follows conventional commit style, making it both clear and specific to the changeset.
Docstring Coverage ✅ Passed Docstring coverage is 94.81% which is sufficient. The required threshold is 80.00%.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch feature/add-pytest.mark.benchmark
📜 Recent review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between d0bf5e7 and cfa4f5b.

📒 Files selected for processing (9)
  • tests/config_edge_test.py (3 hunks)
  • tests/config_fallback_test.py (1 hunks)
  • tests/config_import_test.py (2 hunks)
  • tests/config_test.py (8 hunks)
  • tests/engine_comprehensive_test.py (16 hunks)
  • tests/engine_test.py (62 hunks)
  • tests/main_test.py (21 hunks)
  • tests/rule_builder_test.py (9 hunks)
  • tests/util_test.py (5 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
tests/**

📄 CodeRabbit inference engine (.github/copilot-instructions.md)

Ensure tests run via pytest -v and cover commit, branch, author, and CLI behaviors

Files:

  • tests/engine_comprehensive_test.py
  • tests/config_edge_test.py
  • tests/util_test.py
  • tests/engine_test.py
  • tests/rule_builder_test.py
  • tests/config_import_test.py
  • tests/main_test.py
  • tests/config_fallback_test.py
  • tests/config_test.py
🧠 Learnings (1)
📚 Learning: 2025-10-03T10:28:06.793Z 
Learnt from: CR
PR: commit-check/commit-check#0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-10-03T10:28:06.793Z
Learning: Applies to commit_check/author.py : Validate author email format

Applied to files:

  • tests/engine_test.py
🧬 Code graph analysis (3)
tests/engine_comprehensive_test.py (2)
tests/engine_test.py (2)
  • TestValidationResult (26-31)
  • TestValidationContext (34-49)
commit_check/engine.py (1)
  • ValidationResult (19-23)
tests/engine_test.py (2)
commit_check/engine.py (2)
  • ValidationResult (19-23)
  • BaseValidator (35-94)
tests/engine_comprehensive_test.py (2)
  • TestValidationContext (31-43)
  • TestCommitMessageValidator (46-116)
tests/main_test.py (1)
commit_check/main.py (1)
  • main (127-225)
🪛 Ruff (0.14.0)
tests/main_test.py

27-27: Unused method argument: capfd

(ARG002)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (11)
  • GitHub Check: install (3.12, windows-latest)
  • GitHub Check: install (3.14, macos-latest)
  • GitHub Check: install (3.13, macos-latest)
  • GitHub Check: install (3.13, windows-latest)
  • GitHub Check: install (3.14, windows-latest)
  • GitHub Check: install (3.10, windows-latest)
  • GitHub Check: install (3.10, macos-latest)
  • GitHub Check: install (3.11, windows-latest)
  • GitHub Check: install (3.9, windows-latest)
  • GitHub Check: install (3.9, macos-latest)
  • GitHub Check: Run benchmarks
🔇 Additional comments (9)
tests/config_test.py (1)

12-12: LGTM! Benchmark decorators properly applied.

The @pytest.mark.benchmark decorators are correctly added to all test methods without modifying test logic.

Also applies to: 32-32, 41-41, 61-61, 81-81, 94-94, 109-109, 116-116, 139-139

tests/main_test.py (1)

11-11: LGTM! Benchmark decorators properly applied across all test methods.

The decorators are correctly placed and do not modify any test logic.

Also applies to: 19-19, 26-26, 32-32, 42-42, 55-55, 68-68, 82-82, 96-96, 110-110, 124-124, 134-134, 148-148, 157-157, 169-169, 184-184, 195-195, 211-211, 225-225, 235-235, 249-249, 267-267, 290-290, 303-303

tests/config_edge_test.py (1)

9-9: LGTM! Benchmark decorators correctly applied.

All three test functions are properly decorated with @pytest.mark.benchmark.

Also applies to: 27-27, 50-50

tests/engine_test.py (1)

27-27: LGTM! Comprehensive benchmark decoration across all test methods.

All test methods in this large test file have been properly decorated with @pytest.mark.benchmark without any test logic modifications.

Also applies to: 35-35, 44-44, 53-53, 61-61, 74-74, 87-87, 105-105, 123-123, 145-145, 163-163, 179-179, 193-193, 203-203, 217-217, 231-231, 256-256, 267-267, 279-279, 292-292, 304-304, 319-319, 329-329, 339-339, 356-356, 374-374, 391-391, 409-409, 428-428, 438-438, 450-450, 460-460, 472-472, 486-486, 496-496, 508-508, 519-519, 533-533, 543-543, 555-555, 567-567, 577-577, 587-587, 599-599, 613-613, 628-628, 646-646, 657-657, 670-670, 682-682, 692-692, 705-705, 714-714, 725-725, 740-740, 758-758, 768-768, 781-781, 794-794, 811-811, 821-821, 833-833, 844-844, 855-855

tests/config_fallback_test.py (1)

6-6: LGTM! pytest import and benchmark decorator properly added.

The required pytest import is added and the test function is correctly decorated.

Also applies to: 10-10

tests/engine_comprehensive_test.py (1)

20-20: LGTM! pytest import and benchmark decorators properly added.

The pytest import is added and all test methods are correctly decorated with @pytest.mark.benchmark.

Also applies to: 24-24, 32-32, 47-47, 60-60, 79-79, 101-101, 120-120, 136-136, 156-156, 172-172, 192-192, 208-208, 228-228, 242-242, 270-270, 288-288, 306-306

tests/rule_builder_test.py (1)

5-5: LGTM! pytest import and benchmark decorators properly added.

The pytest import is added and all test methods are correctly decorated.

Also applies to: 9-9, 19-19, 33-33, 45-45, 64-64, 78-78, 99-99, 120-120, 142-142

tests/config_import_test.py (2)

6-6: LGTM!

The pytest import is correctly added to support the decorator usage.

9-9: No changes needed for @pytest.mark.benchmark: The benchmark marker is registered as a no-op in pyproject.toml under tool.pytest.ini_options, so it’s intentionally used for grouping tests.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@codecov
Copy link

codecov bot commented Oct 13, 2025 •
edited
Loading

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87.29%. Comparing base (d0bf5e7) to head (cfa4f5b).
⚠️ Report is 5 commits behind head on main.

Additional details and impacted files 
@@           Coverage Diff           @@
##             main     #308   +/-   ##
=======================================
  Coverage   87.29%   87.29%           
=======================================
  Files           8        8           
  Lines         685      685           
=======================================
  Hits          598      598           
  Misses         87       87

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@codspeed-hq
Copy link

codspeed-hq bot commented Oct 13, 2025

CodSpeed Performance Report

Merging #308 will not alter performance

Comparing feature/add-pytest.mark.benchmark (cfa4f5b) with main (d0bf5e7)

Summary

✅ 27 untouched
🆕 129 new
⏩ 85 skipped1

Benchmarks breakdown

Benchmark BASE HEAD Change
🆕 test_load_config_file_permission_error N/A 331 µs N/A
🆕 test_load_config_invalid_toml N/A 409 µs N/A
🆕 test_tomli_import_fallback N/A 414.6 µs N/A
🆕 test_config_tomli_fallback_direct N/A 1.3 ms N/A
🆕 test_import_paths_coverage N/A 317 µs N/A
🆕 test_tomli_import_fallback_simulation N/A 1.2 ms N/A
🆕 test_default_config_paths_constant N/A 180.2 µs N/A
🆕 test_load_config_default_cchk_toml N/A 360.9 µs N/A
🆕 test_load_config_default_commit_check_toml N/A 356.4 µs N/A
🆕 test_load_config_file_not_found N/A 243.2 µs N/A
🆕 test_load_config_file_not_found_with_invalid_path_hint N/A 322.1 µs N/A
🆕 test_load_config_with_nonexistent_path_hint N/A 226.6 µs N/A
🆕 test_load_config_with_path_hint N/A 383.9 µs N/A
🆕 test_toml_load_function_exists N/A 329.3 µs N/A
🆕 test_tomli_import_fallback N/A 1.4 ms N/A
🆕 test_commit_message_validator_creation N/A 118.5 µs N/A
🆕 test_commit_message_validator_failure N/A 2 ms N/A
🆕 test_commit_message_validator_skip_validation N/A 912.5 µs N/A
🆕 test_commit_message_validator_with_stdin N/A 938 µs N/A
🆕 test_subject_capitalization_fail N/A 1.1 ms N/A
... ... ... ... ...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

Footnotes

  1. 85 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

@shenxianpeng shenxianpeng merged commit bfcc722 into main Oct 13, 2025
32 checks passed
@shenxianpeng shenxianpeng deleted the feature/add-pytest.mark.benchmark branch October 13, 2025 20:20
@shenxianpeng shenxianpeng added tests Add test related changes and removed enhancement New feature or request labels Oct 20, 2025
@coderabbitai coderabbitai bot mentioned this pull request Dec 14, 2025
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

tests Add test related changes

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@shenxianpeng