feat(rule-tester): stricter rule test validations #9086

Vinccool96 · 2024-05-13T14:10:56Z

PR Checklist

Addresses an existing open issue: fixes Enhancement(rule-tester): Stricter rule test validations #9016
That issue was marked as accepting prs
Steps in Contributing were taken

Overview

Brings in changes from eslint/eslint#17654. This is pretty much a copy & paste.

typescript-eslint · 2024-05-13T14:11:00Z

Thanks for the PR, @Vinccool96!

typescript-eslint is a 100% community driven project, and we are incredibly grateful that you are contributing to that community.

The core maintainers work on this in their personal time, so please understand that it may not be possible for them to review your work immediately.

Thanks again!

🙏 Please, if you or your company is finding typescript-eslint valuable, help us sustain the project by sponsoring it transparently on https://opencollective.com/typescript-eslint.

netlify · 2024-05-13T14:11:12Z

✅ Deploy Preview for typescript-eslint ready!

Name	Link
🔨 Latest commit	`3bdc61e`
🔍 Latest deploy log	https://app.netlify.com/sites/typescript-eslint/deploys/6686389c7fa35000084015a2
😎 Deploy Preview	https://deploy-preview-9086--typescript-eslint.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.
Lighthouse	1 paths audited Performance: 98 (🟢 up 4 from production) Accessibility: 100 (no change from production) Best Practices: 92 (no change from production) SEO: 90 (no change from production) PWA: 80 (no change from production) View the detailed breakdown and full score reports

To edit notification comments on pull requests, go to your Netlify site configuration.

nx-cloud · 2024-05-13T14:12:24Z

☁️ Nx Cloud Report

CI is running/has finished running commands for commit 3bdc61e. As they complete they will appear below. Click to see the status, the terminal output, and the build insights.

📂 See all runs for this CI Pipeline Execution

✅ Successfully ran 1 target

nx run repo-tools:postinstall-script

Sent with 💌 from NxCloud.

packages/rule-tester/src/RuleTester.ts

packages/rule-tester/tests/eslint-base/eslint-base.test.js

Vinccool96 · 2024-05-13T15:51:21Z

~~I'm gonna need some help for the failing tests~~

Never mind, there was way less things to fix than I thought.

Vinccool96 · 2024-05-27T14:05:23Z

@auvred Is everything good to merge?

auvred

Left a few minor requests but overall looks great!

Btw, there is no need to ping individuals for review. We have a queue and we get to things when we can. https://typescript-eslint.io/contributing/pull-requests

packages/rule-tester/src/RuleTester.ts

packages/rule-tester/tests/eslint-base/eslint-base.test.js

Vinccool96 · 2024-06-27T12:55:39Z

Sorry for the force pushes, I'm used to do rebase updates

auvred

Did you check this comment? #9086 (comment)

We still miss this assertion

Vinccool96 · 2024-06-27T14:43:24Z

The assertion is here https://github.com/Vinccool96/typescript-eslint/blob/4eebf7df8d89a919fbc03b69c9405d118e9d066a/packages/rule-tester/src/RuleTester.ts#L1047-L1057, lines 1047 to 1057

auvred · 2024-06-27T14:48:44Z

On main there are two different assertions:

typescript-eslint/packages/rule-tester/src/RuleTester.ts

Lines 874 to 884 in de18a6e

    
           const unsubstitutedPlaceholders = 
        
             getUnsubstitutedMessagePlaceholders( 
        
               message.message, 
        
               rule.meta.messages[message.messageId], 
        
               error.data, 
        
             ); 
        
           assert.ok( 
        
             unsubstitutedPlaceholders.length === 0, 
        
             `The reported message has ${unsubstitutedPlaceholders.length > 1 ? `unsubstituted placeholders: ${unsubstitutedPlaceholders.map(name => `'${name}'`).join(', ')}` : `an unsubstituted placeholder '${unsubstitutedPlaceholders[0]}'`}. Please provide the missing ${unsubstitutedPlaceholders.length > 1 ? 'values' : 'value'} via the 'data' property in the context.report() call.`, 
        
           );

typescript-eslint/packages/rule-tester/src/RuleTester.ts

Lines 1032 to 1042 in de18a6e

    
           const unsubstitutedPlaceholders = 
        
             getUnsubstitutedMessagePlaceholders( 
        
               actualSuggestion.desc, 
        
               rule.meta.messages[expectedSuggestion.messageId], 
        
               expectedSuggestion.data, 
        
             ); 
        
           assert.ok( 
        
             unsubstitutedPlaceholders.length === 0, 
        
             `The message of the suggestion has ${unsubstitutedPlaceholders.length > 1 ? `unsubstituted placeholders: ${unsubstitutedPlaceholders.map(name => `'${name}'`).join(', ')}` : `an unsubstituted placeholder '${unsubstitutedPlaceholders[0]}'`}. Please provide the missing ${unsubstitutedPlaceholders.length > 1 ? 'values' : 'value'} via the 'data' property for the suggestion in the context.report() call.`, 
        
           );

Vinccool96 · 2024-06-27T14:51:32Z

Alright, let me plop it back in

Vinccool96 · 2024-06-27T15:07:04Z

The prodigal son has returned

auvred

Thanks for working on this!

JoshuaKGoldberg

The tests for this looked like a real slog, thanks so much for doing this! 🙌

Just a couple of teeny suggestions. Nothing major from me. We can apply them before merging if you don't get a chance before our Monday release.

packages/rule-tester/src/RuleTester.ts

JoshuaKGoldberg

Thanks! 🙌

Vinccool96 · 2024-07-03T18:23:42Z

Had to merge since the main branch had tests that would've failed with the stricter rule tester (tests added by me, what was I thinking!)

Vinccool96 changed the title ~~feat(rule-tester): Stricter rule test validations~~ feat(rule-tester): stricter rule test validations May 13, 2024

auvred requested changes May 13, 2024

View reviewed changes

packages/rule-tester/src/RuleTester.ts Outdated Show resolved Hide resolved

packages/rule-tester/src/RuleTester.ts Show resolved Hide resolved

packages/rule-tester/tests/eslint-base/eslint-base.test.js Outdated Show resolved Hide resolved

auvred added the awaiting response Issues waiting for a reply from the OP or another party label May 13, 2024

Vinccool96 force-pushed the feat/stricter-rule-test-validations branch from aca9d06 to bde1e6e Compare May 13, 2024 15:19

Vinccool96 force-pushed the feat/stricter-rule-test-validations branch from 901db33 to 6e71280 Compare May 13, 2024 20:15

Vinccool96 requested a review from auvred May 13, 2024 21:19

github-actions bot removed the awaiting response Issues waiting for a reply from the OP or another party label May 13, 2024

Vinccool96 force-pushed the feat/stricter-rule-test-validations branch 2 times, most recently from 06d3edf to 5f55140 Compare May 26, 2024 17:09

Vinccool96 force-pushed the feat/stricter-rule-test-validations branch from 5f55140 to cbdf415 Compare May 27, 2024 14:07

bradzacher added the enhancement New feature or request label May 28, 2024

auvred requested changes May 30, 2024

View reviewed changes

auvred added the awaiting response Issues waiting for a reply from the OP or another party label May 30, 2024

Vinccool96 force-pushed the feat/stricter-rule-test-validations branch 3 times, most recently from 7ff0f76 to c13178a Compare June 3, 2024 13:39

Vinccool96 requested a review from auvred June 3, 2024 16:23

github-actions bot removed the awaiting response Issues waiting for a reply from the OP or another party label Jun 3, 2024

Vinccool96 force-pushed the feat/stricter-rule-test-validations branch from a2715a0 to b121617 Compare June 4, 2024 18:02

Vinccool96 mentioned this pull request Jun 4, 2024

Repo: codecov coverage reports are not longer uploaded #9265

Closed

Vinccool96 force-pushed the feat/stricter-rule-test-validations branch 3 times, most recently from 40ec555 to e761ee1 Compare June 6, 2024 14:45

Vinccool96 force-pushed the feat/stricter-rule-test-validations branch from e761ee1 to 271d694 Compare June 17, 2024 13:46

Vinccool96 force-pushed the feat/stricter-rule-test-validations branch from 271d694 to c94ed24 Compare June 25, 2024 14:30

Fixed tests order

4eebf7d

Vinccool96 requested a review from auvred June 27, 2024 13:40

github-actions bot removed the awaiting response Issues waiting for a reply from the OP or another party label Jun 27, 2024

auvred requested changes Jun 27, 2024

View reviewed changes

Added back in forgotten condition

0e08395

Vinccool96 requested a review from auvred June 27, 2024 15:07

Merge branch 'main' into feat/stricter-rule-test-validations

5f0bedc

auvred previously approved these changes Jun 27, 2024

View reviewed changes

auvred added the 1 approval >=1 team member has approved this PR; we're now leaving it open for more reviews before we merge label Jun 27, 2024

JoshuaKGoldberg previously approved these changes Jun 29, 2024

View reviewed changes

packages/rule-tester/src/RuleTester.ts Outdated Show resolved Hide resolved

packages/rule-tester/src/RuleTester.ts Outdated Show resolved Hide resolved

Vinccool96 added 2 commits June 29, 2024 10:52

Merge branch 'refs/heads/main' into feat/stricter-rule-test-validations

9c956d1

Applied suggestions

cc5a70b

Vinccool96 dismissed stale reviews from JoshuaKGoldberg and auvred via cc5a70b June 29, 2024 14:53

Vinccool96 requested a review from JoshuaKGoldberg June 29, 2024 14:53

JoshuaKGoldberg previously approved these changes Jun 29, 2024

View reviewed changes

Vinccool96 added 2 commits July 3, 2024 11:11

Merge branch 'refs/heads/main' into feat/stricter-rule-test-validations

7f201df

Fixed tests

ea7f5f5

Vinccool96 dismissed JoshuaKGoldberg’s stale review via ea7f5f5 July 3, 2024 15:18

Merge branch 'main' into feat/stricter-rule-test-validations

3bdc61e

auvred approved these changes Jul 4, 2024

View reviewed changes

auvred merged commit 7e2b77d into typescript-eslint:main Jul 4, 2024

Vinccool96 deleted the feat/stricter-rule-test-validations branch July 4, 2024 12:35

github-actions bot locked as resolved and limited conversation to collaborators Jul 14, 2024

Uh oh!

feat(rule-tester): stricter rule test validations #9086

feat(rule-tester): stricter rule test validations #9086

Uh oh!

Conversation

Vinccool96 commented May 13, 2024

PR Checklist

Overview

Uh oh!

typescript-eslint bot commented May 13, 2024

Uh oh!

netlify bot commented May 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for typescript-eslint ready!

Uh oh!

nx-cloud bot commented May 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

☁️ Nx Cloud Report

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Vinccool96 commented May 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Vinccool96 commented May 27, 2024

Uh oh!

auvred left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Vinccool96 commented Jun 27, 2024

Uh oh!

auvred left a comment

Choose a reason for hiding this comment

Uh oh!

Vinccool96 commented Jun 27, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

auvred commented Jun 27, 2024

Uh oh!

Vinccool96 commented Jun 27, 2024

Uh oh!

Vinccool96 commented Jun 27, 2024

Uh oh!

auvred left a comment

Choose a reason for hiding this comment

Uh oh!

JoshuaKGoldberg left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

JoshuaKGoldberg left a comment

Choose a reason for hiding this comment

Uh oh!

Vinccool96 commented Jul 3, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

netlify bot commented May 13, 2024 •

edited

Loading

nx-cloud bot commented May 13, 2024 •

edited

Loading

Vinccool96 commented May 13, 2024 •

edited

Loading

Vinccool96 commented Jun 27, 2024 •

edited

Loading