Skip to content

Improves error handling#4

Merged
Jameswlepage merged 3 commits intotest-concurrencyfrom
error-handling
Dec 30, 2025
Merged

Improves error handling#4
Jameswlepage merged 3 commits intotest-concurrencyfrom
error-handling

Conversation

@JasonTheAdams
Copy link
Copy Markdown
Member

Depends on #3

Adds robust error handling to the benchmark runner with clear, formatted terminal output using Rich. The benchmark now fails fast on the first test error and provides detailed debugging information.

Key changes:

  • Stop execution immediately when a test fails instead of continuing with corrupted results
  • Display formatted error panels with test ID, error type, message, and traceback
  • Handle Ctrl+C gracefully with an "Aborted" message
  • Extract display/formatting logic into output.py for cleaner separation of concerns

Error Handling

  • Added TestError exception class that captures test context (ID, type, original error, traceback)
  • Wrapped test processing in try/except blocks with fail-fast behavior
  • Cancel remaining futures when an error occurs
  • Exit with appropriate codes: 1 for test failure, 130 for Ctrl+C

Rich Formatted Output

  • print_test_error() - Detailed error panel with bold headings and colored values
  • print_abort_message() - Yellow panel for Ctrl+C interrupts
  • print_comparison_table() - Multi-model results table
  • print_model_header() / print_results_path() - Status messages

Code Organization

  • Created output.py for all Rich console formatting
  • core.py now focuses purely on test orchestration flow

Visuals

image image

Test Plan

  • Simulated API error (429) triggers error panel and stops execution
  • Ctrl+C displays abort message and exits cleanly
  • Syntax and import verification passes

@Jameswlepage
Copy link
Copy Markdown
Contributor

Errors well 😁

@Jameswlepage Jameswlepage merged commit 17f348c into test-concurrency Dec 30, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants